Weapon usage monitoring system having performance metrics and feedback recommendations based on discharge event detection

ABSTRACT

A system and method for recommending a corrective action based on a performance metric related to a discharge event of a firearm is provided. The method includes receiving, by a first event detection module associated with a first firearm, a plurality of first input signals over a sample window of time from a first inertial measurement unit configured on the first firearm; identifying, by the first event detection module, an occurrence of a first shot discharge of the first firearm at a shot time based on the plurality of first input signals; determining a first orientation of the first firearm at the shot time; determining a second orientation of the first firearm at a first time before the shot time; comparing the first and second orientations; determining a first performance metric related to pre-shot weapon orientation; and displaying a first recommended corrective action based on the first performance metric.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.18/143,404 filed May 4, 2023, which claims priority to U.S. ProvisionalApplication No. 63/455,852 filed on Mar. 30, 2023. U.S. application Ser.No. 18/143,404 filed May 4, 2023 is a continuation of InternationalApplication No. PCT/US2022/023027 filed on Apr. 1, 2022, which claimsthe benefit of U.S. Patent Application Nos. 63/169,283 filed on Apr. 1,2021 and 63/216,037 filed on Jun. 29, 2021. U.S. application Ser. No.18/143,404 filed May 4, 2023 is a continuation of U.S. patent Ser. No.17/524,302 filed on Nov. 11, 2021, which is a continuation of U.S.patent application Ser. No. 16/995,990, filed Aug. 18, 2020, andpublished on Dec. 3, 2020 as U.S. Patent Publication 2020/0378708 whichis a bypass continuation of International Patent Application No.PCT/US2019/055925, filed Oct. 11, 2019, and published on Apr. 16, 2020,as Publication No. WO/2020/077254, which claims the benefit of U.S.Provisional Patent Application Ser. No. 62/745,028, filed Oct. 12, 2018.U.S. patent Ser. No. 17/524,302 is also a continuation of U.S. patentapplication Ser. No. 16/599,976 filed Oct. 11, 2019, and published onApr. 16, 2020 as U.S. Patent Publication 2020/0117900. U.S. applicationSer. No. 18/143,404 filed May 4, 2023 is a continuation of U.S. patentSer. No. 17/524,302 is also a continuation of U.S. patent applicationSer. No. 16/460,348 filed Jul. 2, 2019, and published on Jan. 2, 2020 asU.S. Patent Publication 2020/0003512, which is a bypass continuation ofInternational Patent Application No. PCT/US2018/015614, filed Jan. 27,2018, and published on Aug. 2, 2018, as Publication No. WO/2018/140835,which claims the benefit of U.S. Provisional Patent Application Ser. No.62/451,620, filed Jan. 27, 2017. Each of the above-identifiedapplications are hereby incorporated by reference as if fully set forthin its entirety.

BACKGROUND

Typically, firearm tracking systems have been very limited, oftenrequiring complex manufacturing steps in order to enable a determinationof whether a weapon has been used. These systems typically have issueswith reliability, have poor performance (e.g., short battery life), lackthe ability to add new features, and suffer other limitations.Separately, systems for providing remote support to firearm users arealso typically very limited. For example, a remote support usermonitoring a deployment of firearm users within a deployment location,such as a combat zone, relies on the information reported to him or herin order to make appropriate decisions regarding providing support forthose users. However, these conventional systems require a remotesupport user to manually analyze information about the firearm users andto manually determine how to support those firearm users, which may, inat least some cases, take more time than is available. For example,during an active fire fight between firearm users and hostilecombatants, the amount of time it takes to determine to deployreinforcements, deliver additional ammunition, or otherwise support thefirearm users can dictate the success of the engagement. Accordingly, aneed exists for improved systems that involve recording and trackingactivities of individuals, including more advanced methods and systemsfor tracking discharges from firearms and more advanced methods formonitoring conditions of firearms, other assets, and users within adeployment location and automating actions to perform for remotelysupporting those firearm users, such as in preparation for, during,and/or after an engagement with a hostile threat.

SUMMARY

A method for recommending a corrective action based on a performancemetric related to a discharge event of a firearm is provided. The methodincludes receiving, by a first event detection module associated with afirst firearm of a first user, a plurality of first input signals over asample window of time from a first inertial measurement unit (IMU)configured on the first firearm; identifying, by the first eventdetection module, an occurrence of a first shot discharge of the firstfirearm at a shot time based on the plurality of first input signals;determining a first orientation of the first firearm at the shot time;determining a second orientation of the first firearm at a first timebefore the shot time; comparing the first and second orientations;determining a first performance metric related to pre-shot weaponorientation; and displaying a first recommended corrective action to thefirst user based on the first performance metric.

In embodiments, the method further comprises determining a thirdorientation of the first firearm at a second time after the shot time;comparing the first and third orientations; determining a secondperformance metric related to post-shot weapon orientation; displaying asecond recommended corrective action to the first user based on thesecond performance metric. The method can further include establishing ahistorical baseline performance metric of the first user based oncollected historical shot detection information about the first user;comparing the first performance metric with the historical baselineperformance metric; and displaying the comparison. Receiving theplurality of first input signals can include receiving a firstacceleration input signal along a first axis, a second accelerationsignal along a second axis and a third acceleration input signal along athird axis; and receiving a first rotation input signal around the firstaxis, a second rotation input signal around the second axis and a thirdacceleration input signal around the third axis.

In embodiments, the method can further include receiving, by a secondevent detection module associated with a second firearm of a seconduser, a plurality of second input signals over a sample window of timefrom a second IMU configured on the second firearm; identifying, by thesecond event detection module, an occurrence of a first shot dischargeof the second firearm at a shot time based on the plurality of secondinput signals; determining a first orientation of the second firearm atthe shot time; determining a second orientation of the second firearm ata first time before the shot time; comparing the first and secondorientations; and assigning a third performance metric to the seconduser based on the first and second orientations of the second firearm.

In examples, the method can further include ranking a performance of thefirst user relative to the second user based on the first and thirdperformance metrics; and providing visual feedback to the first user ofthe ranking. Providing the visual feedback can comprise displaying theranking on an augmented reality headset. The first and third performancemetrics can be assigned to a first group and assigning a first aggregatestability index score.

In other examples, the method can include determining a fourthperformance metric for a third user; determining a fifth performancemetric for a fourth user; combining the fourth and fifth performancemetrics to a second group and assigning a second aggregate performanceindex score; comparing the first aggregate performance index score withthe second aggregate performance index score; and determining a rankedperformance of the first group versus the second group based on thecomparing. The plurality of first input signals can include accelerationand rotation input signals. Identifying the occurrence of the first shotcan include assigning respective acceleration and rotation input signalsto sample event candidates at respective windows of time; comparing theacceleration signals of the sampled event candidates to a dischargeacceleration template that represents a confirmed weapon dischargeevent; identifying a subset of accepted accelerations from the sampledevent candidates that satisfy the discharge acceleration template;comparing a first rotation input signal from the sample window of timeassociated with each accepted acceleration in the subset to a firstrotation template that represents a confirmed weapon discharge event;and determining whether the sampled event candidate is a discharge eventbased on satisfying both the discharge acceleration template and thefirst rotation template.

A system for determining a performance metric based on a discharge eventof a firearm can include a first inertial measurement unit (IMU), afirst event detection module and a display. The first IUM can bedisposed on a first firearm that senses movement including at least oneof accelerations and rotations. The first event detection module canreceive a plurality of first input signals from the first IMU indicativeof the movement and be configured to: identify an occurrence of a firstshot discharge of the first firearm at a shot time based on theplurality of first input signals; determine a first orientation of thefirst firearm at the shot time; determine a second orientation of thefirst firearm at a first time before the shot time; compare the firstand second orientations; and determine a first performance metricrelated to pre-shot weapon orientation. The display can display thefirst recommended corrective action.

In examples, the first event detection module can further be configuredto: determine a third orientation of the first firearm at a second timeafter the shot time; compare the first and third orientations; determinea second performance metric related to post-shot weapon orientation; anddisplay a second recommended corrective action to the first user basedon the second performance metric.

In examples, the first event detection module is further configured toestablish a historical baseline performance metric of the first userbased on collected historical shot detection information about the firstuser; and compare the first performance metric with the historicalbaseline performance metric. The comparison is shown in the display. Theplurality of first input signals can include a first acceleration inputsignal along a first axis, a second acceleration signal along a secondaxis and a third acceleration input signal along a third axis; and afirst rotation input signal around the first axis, a second rotationinput signal around the second axis and a third acceleration inputsignal around the third axis.

In other examples, the system can further include a second IMU and asecond event detection module. The second IMU can be disposed on asecond firearm that senses movement including at least one ofaccelerations and rotations. The second event detection module can beassociated with the second firearm that receives a plurality of secondinput signals from the second IMU indicative of the movement. The secondevent detection module is configured to: identify an occurrence of afirst shot discharge of the second firearm at a shot time based on theplurality of second input signals; determine a first orientation of thesecond firearm at the shot time; determine a second orientation of thesecond firearm at a first time before the shot time; compare the firstand second orientations; and assign a third performance metric to thesecond user based on the first and second orientations of the secondfirearm.

In embodiments, the first and third performance metrics can be compared.The display can show the comparison. The display can include a usagemonitoring system including one of a virtual reality and augmentedreality system, wherein the first performance metric is displayed on theusage monitoring system. The first and third stability indexes arecombined to a first group and assigned a first aggregate performancemetric score. The system can further be configured to: determine afourth stability index for a third user; determine a fifth stabilityindex for a fourth user; combine the fourth and fifth stability indexesto a second group and assigning a second aggregate stability index scoreto the second group; compare the first aggregate performance metricscore with the second aggregate performance metric score; determine aranked performance of the first group versus the second group based onthe comparing. The display shows the ranked performance.

In examples, the plurality of first input signals include accelerationand rotation input signals. The first event detection module isconfigured to: assign respective acceleration and rotation input signalsto sample event candidates at respective windows of time; compare theacceleration signals of the sampled event candidates to a dischargeacceleration template that represents a confirmed weapon dischargeevent; identify a subset of accepted accelerations from the sampledevent candidates that satisfy the discharge acceleration template;compare a first rotation input signal from the sample window of timeassociated with each accepted acceleration in the subset to a firstrotation template that represents a confirmed weapon discharge event;and determine whether the sampled event candidate is a discharge eventbased on satisfying both the discharge acceleration template and thefirst rotation template.

BRIEF DESCRIPTION OF THE FIGURES

The detailed description of some embodiments of the present disclosureis made below with reference to the accompanying figures, wherein likenumerals represent corresponding parts of the figures.

FIG. 1 is a diagrammatic view of a firearm monitoring and remote supportsystem in accordance with embodiments of the present disclosure.

FIG. 2 is a side perspective view of a firearm in use in accordance withthe embodiments of the present disclosure.

FIG. 3 is a diagrammatic view of an internal configuration of acomputing device in accordance with embodiments of the presentdisclosure.

FIG. 4 is a diagrammatic view of various system sub-components ofsoftware used for firearm monitoring and remote support in accordancewith embodiments of the present disclosure.

FIG. 5 is a diagrammatic view of various inputs, processing options, andoutputs for threat detection and analysis in accordance with embodimentsof the present disclosure.

FIGS. 6-11 are illustrations of GUIs of software of a firearm monitoringand remote support system in accordance with embodiments of the presentdisclosure.

FIG. 12 is an illustration of a mesh network system in accordance withembodiments of the present disclosure.

FIG. 13 is a diagrammatic view of various sub-components of a connectionpoint in accordance with embodiments of the present disclosure.

FIG. 14 is a flowchart showing a technique for threat detection andresponse in accordance with embodiments of the present disclosure.

FIG. 15 is a flowchart showing a technique for GUI visualization ofsensor and threat information in accordance with embodiments of thepresent disclosure.

FIG. 16 is a flowchart showing a technique for compression andcollection of information in accordance with embodiments of the presentdisclosure.

FIG. 17 is an illustration of gestures, positions and locations of afirearm indicative of or in preparation for live fire in accordance withembodiments of the present disclosure.

FIGS. 18-19 are illustrations of multiple users and assets engaged inlive fire in accordance with embodiments of the present disclosure.

FIG. 20 is a bottom front perspective view of a firearm including afirearm usage monitoring system in accordance with the embodiments ofthe present disclosure.

FIG. 21 is a top rear perspective view of the firearm of FIG. 20 .

FIG. 22 is an exploded view of the firearm of FIG. 20 .

FIG. 23 is a perspective view of first and second grip panels of thefirearm and the firearm usage monitor in accordance with embodiments ofthe present disclosure.

FIG. 24 is an electrical schematic view of the firearm usage monitoringsystem in accordance with embodiments of the present disclosure.

FIG. 25 and FIG. 26 are schematic views of the firearm usage monitoringsystem in accordance with embodiments of the present disclosure.

FIGS. 27, 28, 29 and 30 are diagrammatic views of various systemsub-components for the firearm usage monitoring system in accordancewith embodiments of the present disclosure.

FIG. 31 is a partial perspective view of a firearm including the firearmusage monitoring system in accordance with embodiments of the presentdisclosure.

FIG. 32 is a process view of a machine control system of the firearmusage monitoring system in accordance with embodiments of the presentdisclosure.

FIGS. 33 and 34 are diagrammatic views of various system sub-componentsfor the firearm usage monitoring system in accordance with embodimentsof the present disclosure.

FIGS. 35, 36, 37, 38, 39, 40, and 41 are flowchart showing techniquesfor communicating with and monitoring firearms and user of thosefirearms in accordance with embodiments of the present disclosure.

FIGS. 42A-42D are side views of exemplary firearms that include eventdetection modules in accordance with embodiments of the presentdisclosure.

FIG. 43 is a bottom perspective view of a grip module that incorporatesan event detection module used in the exemplary firearms shown in FIGS.42A-42D.

FIG. 44 is an exploded perspective view of the grip module of FIG. 43 .

FIG. 45 is a front perspective view of a barrel assembly thatincorporates an event detection module in accordance with furtherembodiments of the present disclosure.

FIG. 46A is an exploded perspective view of another exemplary firearmthat incorporates an event detection module in accordance with furtherembodiments of the present disclosure.

FIG. 46B is a front perspective view of the event detection module ofFIG. 46A.

FIG. 47 is an exploded perspective view of another exemplary firearmthat incorporates an event detection module in accordance with furtherembodiments of the present disclosure.

FIG. 48A is side view of another exemplary firearm that incorporates anevent detection module in accordance with further embodiments of thepresent disclosure.

FIG. 48B is a front view of a spade grip of the exemplary firearm ofFIG. 48A that incorporates the event detection module.

FIG. 49 is an exploded perspective view of a grip module of the eventdetection module incorporated on the firearm of FIG. 48A.

FIG. 50 is a partial cutaway bottom perspective view of an eventdetection module that incorporates a trigger pull sensor assembly inaccordance with further embodiments of the present disclosure.

FIG. 51A is side view of the firearm shown in FIG. 50 .

FIG. 51B is a side view of the event detection module and grip housingremoved from the firearm.

FIG. 52 is a side view of an exemplary trigger assembly.

FIG. 53A is a schematic diagram of the trigger pull sensor assemblyshown with a contact sensor in a depressed position corresponding to thetrigger in a ready, not-pulled position.

FIG. 53B is a schematic diagram of the trigger pull sensor assembly ofFIG. 53A shown with the contact sensor in an extended positioncorresponding to the trigger in an actuated, pulled position.

FIG. 54A is side view of an event detection module that incorporates asafety selector switch sensor and safety selector switch in accordancewith further embodiments of the present disclosure, the safety selectorswitch shown in a safety position.

FIG. 54B is side view of the event detection module of FIG. 54A shownwith the safety selector switch in a use position.

FIG. 54C is a diagrammatic view of an event detection module inaccordance with embodiments of the present disclosure.

FIG. 55 is a front perspective view of an exemplary firearm illustratingvarious axes of inputs.

FIG. 56A is a first plot of various sample event candidates shown asacceleration vector magnitudes over time.

FIG. 56B is a second plot representing matches of selected sample eventcandidates to a discharge acceleration template.

FIG. 56C is a third plot representing the matches of the second plot.

FIG. 57 is an example logic flow of a machine learning algorithm used bythe event detection module.

FIG. 58A is a plot illustrating a sample shot profile (dotted line) anda baseline shot profile (solid line) before alignment.

FIG. 58B is a plot illustrating the sample shot profile and the baselineshot profile of FIG. 58A and shown after alignment.

FIG. 58C is a plot illustrating a signal or sample shot profile over atime window having three identifiable acceleration peaks including afirst peak, a second peak and a third peak 4658.

FIGS. 59A-59C illustrate histograms of shot separation for a newfirearm, a medium firearm and an old firearm, respectively.

FIG. 60 is an example logic flow of a method of tracking a split time ofa firearm.

FIG. 61 is an example logic flow of a method of tracking a stability ofa firearm during a discharge event.

FIGS. 62, 63, and 64 depict real-time data processing of weapon responsedevice data from a deployment location.

FIGS. 65 and 66 depict the replaying of weapon response device data froma deployment location.

FIGS. 67 and 68 depict sensor hub aggregation within the weaponmonitoring system.

FIG. 69 depicts mission monitoring within the weapon monitoring system.

FIG. 70 depicts deployment of augmented reality and virtual realitysystems within the weapon monitoring system.

FIG. 71 depicts an example of a disconnected networking solution thatprovides near-real time situational awareness data on a local areanetwork.

FIG. 72 depicts an example of an edge networking system providingsituational awareness of deployed assets in communication with edgecomputing resources.

DETAILED DESCRIPTION

By way of example, and referring to FIG. 1 , an embodiment of firearmmonitoring and remote support system 100 includes firearm monitoring andremote support application 102 which processes signals received from oneor more of firearms 104, wearable devices 106, or stationary devices 108to detect and assess threats against users of firearms 104. For example,the signals received from firearms 104, wearable devices 106, and/orstationary devices 108 can be processed to determine whether and how torespond to a threat against the users of firearms 104, including byautomating a deployment of response infrastructure 110 to a location ofor proximate to the users of firearms 104. Application 102 is run,executed, interpreted, or otherwise operated at server device 112, whichcommunicates, directly or indirectly, with firearms 104, wearabledevices 106, and/or stationary devices 108 using network 114 andconnection point 116.

The application 102 is software for monitoring users within a deploymentlocation. The users are humans or non-human entities (e.g., mobile orstationary robots). The users operate firearms 104 and wear wearabledevices 106. In embodiments, the users may operate stationary devices108. Alternatively, stationary devices 108 may be operated withoutaction by the users. A user may be mobile or stationary, for example,based on whether they are human or non-human and/or based on a directiveof the user. For example, a user who operates a sniper rifle or otherheavy powered weaponry or machinery may in some cases be considered astationary user. The deployment location is a geographic regionincluding one or more terrain types and may be wholly developed (e.g., acity or other urban environment area), partially developed (e.g., arelatively small or rural living area), or wholly undeveloped (e.g., amountainous, forested, desert, or other natural area). In particular,the deployment location represents a location to which one or more usersare deployed. For example, the one or more users may be deployed toidentify, address, or otherwise neutralize a hostile threat. In anotherexample, the one or more users may be deployed to rescue hostages orotherwise assist civilians or friendly forces.

The signals received from firearms 104, wearable devices 106, and/orstationary devices 108 represent sensor information measured forfirearms 104, wearable devices 106, and/or stationary devices 108.Firearms 104 include sensors 118, wearable devices 106 include sensors120, and stationary devices 108 include sensors 122. Sensors 118,sensors 120, and sensors 122 include hardware sensors used to measureaspects of firearms 104, wearable devices 106, and stationary devices108, respectively. For example, sensors 118, sensors 120, and sensors122 may each include one or more of an accelerometer, a gyroscope, amagnetometer, a geolocation sensor, a moisture sensor, a pressuresensor, or the like. Sensors 118, sensors 120, and sensors 122 may eachbe embodied in inertial measurement units included within or otherwisecoupled to firearms 104, wearable devices 106, and stationary devices108, respectively. In embodiments, sensors 118, sensors 120, and sensors122 may each include the same sensors. In embodiments, sensors 118,sensors 120, and sensors 122 may include partially or wholly differentsensors.

The signals received from firearms 104, wearable devices 106, and/orstationary devices 108 are processed to monitor the status of firearms104, wearable devices 106, and/or stationary devices 108. Application102 can monitor the status of firearms 104, wearable devices 106, and/orstationary devices 108 by using the signals to update position and/ororientation information for firearms 104, wearable devices 106, and/orstationary devices 108, and/or for users thereof. The updated positionand/or orientation information can provide details regarding current useof firearms 104, wearable devices 106, and/or stationary devices 108,for example, to indicate use states of firearms 104, wearable devices106, and/or stationary devices 108 and/or to indicate how firearms 104,wearable devices 106, and/or stationary devices 108 are being usedwithin the deployment region.

Monitoring the status of firearms 104 may include generating and/orupdating information for visualizing or otherwise representing a cone offire for firearms 104. A cone of fire, or cone, is or refers to anexpected area of potential fire for a firearm 104. The endpoint of thesector of a cone represents a current location of a firearm 104. Theremaining portion of the cone represents a potential area which,provided the firearm 104 remains stationary at the location representedby the endpoint of the sector), projectiles from the firearm 104 may befired. The cones for firearms 104 may be visually represented byapplication 102, for example, within one or more GUIs generated andoutput by application 102. In embodiments, the size and layout of a conecan be defined based on one or both of the type of a firearm 104corresponding to the cone or the skill of the user of the firearm 104.In embodiments, the size and layout of the cone can be determined usingthe errors in measurements from the IMU and GPS to represent thepotential locations in which the projectile from the firearm may impact.By way of these examples, the shape of the cone of fire can bearbitrarily capped by the effective range of fire for the firearm andthe round being used. In embodiments, the cone of fire can then becapped or otherwise set to a predetermined size and shape by thepre-determined skill rating associated with the skill of the user. Inembodiments, larger caliber firearms may have an increased effectiverange of fire. As such, the bullet itself can have the potential to gowell beyond the drawn cone of fire. For example, a larger firearm mayhave a longer cone than a smaller firearm. In another example, a skilleduser who is capable of accurate marksmanship may have a smaller (e.g.,narrower) cone than one who is less accurate, such as because theskilled user is statistically expected to more accurately hit a target.In yet another example, where learning models (e.g., of a machinelearning system) determine that the user tends to fire too much to theleft or right, the cone for that user can be accordingly projected. Theapplication 102 monitors the status of firearms 104 including byperforming real-time updates to cones corresponding to firearms 104. Forexample, where a GUI of application 102 visually represents users withina deployment location and shows cones, application 102 can automaticallyupdate locations and orientations of the cones, for example, based onsignals received from firearms 104.

The signals received from firearms 104, wearable devices 106, and/orstationary devices 108 are further processed to detect threats withinthe deployment region, including by analyzing whether and/or how torespond to those detected threats. Application 102 can detect threatswithin the deployment region by using the signals received from firearms104, wearable devices 106, and/or stationary devices 108 to determinewhether users thereof are exposed to a threat or may become exposed to athreat. For example, the signals may be used to determine that firearms104 have been drawn or otherwise moved into a readied position, forexample, to prepare to engage a threat. In another example, the signalsmay be used to determine that firearms 104 are actively engaging athreat, for example, based on a detected firing of firearms 104 and/orbased on a coalescence of cones of multiple firearms 104. In yet anotherexample, the signals may be used to determine that ammunition suppliesfor some or all firearms 104 are running low or depleted. In yet anotherexample, the signals may be used to automate a response to the threat,for example, by deploying reinforcements to assist in engaging thethreat, by deploying additional ammunition resources to the deploymentlocation, or otherwise.

The threat may be a human or non-human (e.g., robotic, vehicular,non-human animal, etc.) hostile which presents or may present a risk ofharm to users of firearms 104, wearable devices 106, and/or stationarydevices 108. For example, the threat may be one or more enemy combatantswho possess weapons or other means to present a risk of harm to theusers of firearms 104, wearable devices 106, and/or stationary devices108, to civilians, or to other persons or assets friendly to the usersof firearms 104, wearable devices 106, and/or stationary devices 108. Inanother example, the threat may be one or more robots or animals trainedto attack the users of firearms 104, wearable devices 106, and/orstationary devices 108. The threat may alternatively be or refer to acondition or situation which presents a risk of harm to the users offirearms 104, wearable devices 106, and/or stationary devices 108, tocivilians, or to other persons or assets friendly to the users offirearms 104, wearable devices 106, and/or stationary devices 108. Forexample, the threat may be or relate to a terrain element which presentsa risk of bodily harm or obstructs a traveling path of the users offirearms 104, wearable devices 106, and/or stationary devices 108. Insome cases, the threat may refer to terrain elements which are naturallyoccurring. In other cases, the threat may refer to terrain elementswhich present a risk of harm or obstruction because of actions taken bya hostile.

In response to a detected threat, application 102 may in some casescause a deployment of response infrastructure 110 to the deploymentlocation. Response infrastructure 110 includes or otherwise refers toassets or personnel used to assist in addressing the detected threat.For example, response infrastructure 110 may be or include unmannedaerial vehicles (UAVs) or other aircraft. The UAVs or other aircraft maybe configured to drop ammunition re-supplies within the deploymentlocation, for example, in response to application 102 determining thatcurrent ammunition supplies of one or more users of firearms 104 arerunning low or depleted before, during, or after an engagement with adetected threat. In another example, response infrastructure 110 may beor include transport vehicles used to transport reinforcements withinthe deployment location, for example, in response to application 102determining that additional manpower is required or would be beneficialfor engaging the detected threat. Response infrastructure 110 may bedeployed to a location of connection point 116, for example, which maybe known or determined using a geolocation sensor included within orotherwise coupled to connection point 116. Alternatively, a differentlocation to which response infrastructure 110 is deployed may bedetermined by application 102.

In embodiments, response infrastructure 110 may refer to components,assets, or other matter rather than to specific infrastructure used totransport or otherwise deploy those components, assets, or other matterwithin the deployment location. For example, response infrastructure 110may refer to firearms, ammunition, medical equipment, or other assetswhich can be deployed using a UAV, another aircraft, or another deliverymechanism. In embodiments, response infrastructure 110 may refer tolocations, components, assets, or other matter which may not travel tothe deployment location. For example, response infrastructure 110 mayinclude or otherwise refer to one or more locations at which assetinventories (e.g., firearm, ammunition, medical, or other inventorystocks) are stored and/or to hardware or other machinery or assets atthose locations.

Application 102 may process the signals received from firearms 104,wearable devices 106, and/or stationary devices 108 against informationstored within database 124 to monitor firearms 104, wearable devices106, and/or stationary devices 108 and/or to detect and analyze athreat. Database 124 stores information relating to firearms 104,wearable devices 106, and/or stationary devices 108. For example, theinformation relating to a firearm 104 stored within database 124 mayinclude information about the firearm type, maximum amount of ammunitionwithin a magazine, firing rate, maximum firing range, maintenancestatus, sensors included or coupled, or the like. Database 124 may alsostore information indirectly relating to a firearm 104, for example,information relating to ammunition types, inventory information (e.g.,in a stockpile or warehouse from which reserves can be deployed for usein response to a detected threat), connected or connectable devices(e.g., wearable devices 106), or the like. Database 124 may also storeinformation relating to users of firearms 104, for example, userinformation including names, ranks, years of service, skill levels,notable achievements, numbers of deployments, numbers of engagements,weapons currently possessed in the deployment location, ammunitionstocks present in the deployment location, numbers of shots fired sincearrival at the in the deployment location, health information, threatengagement information, or the like. In embodiments, information storedwithin database 124 relating to firearms may be retrieved frommanufacturers, distributors, or other vendors of those firearms. Forexample, where access is available, application programming interface(API) calls can be made to retrieve the information from externalsystems which the manufacturers, distributors, or other vendors use tostore such information. The information stored within database 124 maybe included in a knowledgebase accessed by application 102. For example,the knowledgebase can represent a collection of knowledge associatedwith assets used by or with system 100, for example, for detecting andanalyzing threats.

Connection point 116 is used to facilitate communications betweenfirearms 104, wearable devices 106, and/or stationary devices 108 andnetwork 114. Network 114 may be a network of computers (e.g., a localarea network (LAN), a wide area network (WAN), a virtual private network(VPN), a peer-to-peer (P2P) network, or an intranet), or a network ofnetworks (e.g., the Internet), or another network (e.g., a cellularnetwork). Connection point 116 is a device configured to communicateover network 114. Connection point 116 may communicate with firearms104, wearable devices 106, and/or stationary devices 108 over Ethernet,transmission control protocol (TCP), Internet protocol (IP), power linecommunication, Wi-Fi, Bluetooth®, infrared, radio frequency (RF),general packet radio services (GPRS), global system for mobilecommunications (GSM), frequency-division multiple access (FDMA),code-division multiple access (CDMA), evolution-data optimized (EVDO),Z-Wave, ZigBee, 3G, 4G, 5G, another protocol, or a combination thereof.In embodiments, connection point 116 may be a router, beacon, wirelessconnection point (e.g., a Wi-Fi connection point), lighting system,camera, or other network-connected devices.

In embodiments, connection point 116 may be one of a number ofconnection points deployed within the deployment location. For example,each connection point may be configured to facilitate communications forcertain ones of firearms 104, wearable devices 106, and/or stationarydevices 108. In another example, bandwidth limitations or otherconstraints may reduce the connection strength or status betweenconnection point 116 and ones of firearms 104, wearable devices 106,and/or stationary devices 108, in which case other connection pointslocated elsewhere in the deployment location may be leveraged forredundancies and back-up communication mechanisms.

In embodiments, connection point 116 may be included in or otherwise usea mesh network to facilitate communications between server device 112and one or more of firearms 104, wearable devices 106, or stationarydevices 108 over network 114. The mesh network may be or represent anetwork of connections between firearms 104, wearable devices 106,stationary devices 108, connection points (e.g., connection point 116),and/or other devices, such as response infrastructure 110, mobilerobots, or the like. The mesh network may form part of a large meshnetwork, allowing devices, such as firearms and mobile robots, tocommunicate directly with one another, rather than having to firstconnect through a centralized network communication hub, or as asupplement to communication by one or more devices to such a hub.

In embodiments, application 102 processes signals received from assetsother than firearms 104, wearable devices 106, and/or stationary devices108. For example, instead of or in addition to signals received fromfirearms 104, wearable devices 106, and/or stationary devices 108,application 102 can process signals received from one or more ofvehicles, mortars, and/or other trackable assets. Each of the vehicles,mortars, and/or other trackable assets may include one or more sensors,which may be the same or different from one or more of sensors 118,sensors 120, and/or sensors 122.

In embodiments, some or all users within a deployment location may beunderground. In such a case, system 100 can use geolocation systems(e.g., a global navigation satellite system, for example, the globalpositioning system (GPS), the global navigation satellite system(GLONASS), the BeiDou navigation satellite system (BDS), Galileo, or thelike) to track subterranean locations of users. In some suchembodiments, assets such as body cameras, heads-up displays, or the likemay be used to supplement subterranean tracking of users.

In embodiments, server device 112 may be part of a cloud computinginfrastructure. For example, application 102 may be or representfunctionality of a software-as-a-service (SaaS) or platform-as-a-service(PaaS) cloud system. In such embodiments, application 102 may be asingle- or multi-instance software application run using one or more webservers, application servers, hypervisors, or the like. In suchembodiments, server device 112 may be or include a hardware server(e.g., a computing device), a software server (e.g., a web server and/ora virtual server), or both. For example, where server device 112 is orincludes a hardware server, server device may be a computing devicelocated in a rack, such as of a data center.

In embodiments, connection point 116 may use or otherwise include anefficient architecture and components for low power consumption,including energy harvesting mechanisms, such as harvesting the energy ofmotion of firearms 104 and/or wearable devices 106 or energy from therecoil of firearms 104 to provide power for storage and/or reporting ofdata to the application 102. The energy harvesting mechanisms may alsobe configured to harvest local energy in the RF domain or otherappropriate local electromagnetic signals of sufficient strength.

In embodiments, sensors 120 of wearable devices 106 may include orotherwise integrate with physiological monitors. A heart rate band ormonitor can be an indicator of a distressed situation creating anotification. In embodiments, wearable devices 106 may integrate theEmergency Response Data communications architecture. In embodiments,wearable devices 106 may include body cameras which capture imagesand/or video. In such embodiments, sensors 120 of wearable devices 106may include image sensors.

In embodiments, application 102 generates geofence-based alerts. Forexample, the geofence capability can be implemented around a warehousewhere weapons are stored to track weapons for inventory control orthreatening situations. In another example, the geofence capability canbe implemented around a central area within the deployment location, forexample, the connection point 116.

In embodiments, application 102 integrates with mobile devicetechnology. Application 102 can send critical messages in a timelymanner, such as through an app installed on a mobile phone or othermobile computing devices of a user of firearm 104. The app may bedirectly connected to dispatchers, such as allowing the caller torequest assistance.

Referring to FIG. 2 , firearm 200 is one of firearms 104 used inconnection with system 100 and user 202 is the user of firearm 200. Inthe example shown, firearm 200 is depicted as an assault rifle. However,firearms which may be used in connection with a firearm monitoring andremote support system in accordance with the embodiments of thisdisclosure may be of other firearm types. For example, types of thefirearms 104 other than assault rifles may include, but are not limitedto, pistols, revolvers, shotguns, other rifles, or the like. Althoughthe following discussion regarding firearm 200 is with respect to thestructure of an assault rifle particularly, similar discussion withrespect to other firearm types is understood, and it will be understoodfrom the following discussion how sensors may be used with other firearmtypes.

In particular, the illustration of FIG. 2 is intended to describelocations of firearm 200 at which various sensors or other componentsmay be included or coupled. Examples of sensors or other componentswhich may be included in or coupled to the various locations showninclude, but are not limited to, an IMU (e.g., including anaccelerometer and/or a gyroscope), a geolocation sensor, a forceconnector, a power input, a battery charger, a laser, a regulator, aserial communication system component, a flash memory, a networkinterface, a programmable hardware unit, or the like.

Firearm 200 includes one or more structures for performing orfacilitating operations typical of a firearm, for example, for storingammunition, firing one or more projectiles from the ammunition,controlling the storage and firing of ammunition, and more. Inembodiments, firearm 200 can include an action structure, a stockstructure, and a barrel structure. In embodiments, firearm 200 caninclude one or more rails. A rail may, for example, be located on one ormore of, or proximate to one or more of, the action structure, the stockstructure, or the barrel structure.

The action structure is or refers to the structure of components whichare used to handle and propel ammunition during firing. For example, theaction structure may include one or more components which are used toload, lock, fire, extract, and/or eject ammunition or shells thereof.Depending on the particular type of firearm, the action structure mayuse a break action mechanism, a bolt action mechanism, a lever actionmechanism, or another action mechanism. The action structure may includea charging handle used to move a hammer to a ready position for firing.The action structure may include a forward assist component that moves abolt fully forward in the event a return spring fails to do so. Theaction structure may include a gas operating system which directs energyfor operating a locked breech of the action mechanism. The actionstructure may include a hammer that strikes a firing pin or othercomponent of the action mechanism to cause the combustion or compressionwhich fires a projectile from the barrel structure of the firearm. Theaction structure may include an ejection port which uses forced gas orother energy resulting from the combustion or compression to eject anammunition shell from the barrel structure of the firearm after theprojectile thereof has been fired. The action structure may also includecomponents other than those described above.

The stock structure is or refers to a structure of components whichprovide support to the action structure and/or to the barrel structure.In embodiments, the stock structure includes a butt and a fore-end. Thebutt and the fore-end may be included in a one-piece stock structure orin a two-piece stock structure. The butt includes a grip and a comb. Thegrip is a component which may be held by a user of the firearm duringthe operation of the firearm. The comb is a portion of the butt whichsupports a portion of a body of the user of the firearm during theoperation of the firearm. A hook may be coupled to the butt of the stockstructure, for example, to support a portion of a body of the user ofthe firearm during the operation of the firearm. The butt may be solid.Alternatively, the butt may be collapsible or telescoping. The fore-endmay include a handguard for protecting a hand of a user of the firearmfrom heat generated at the barrel structure of the firearm during theoperation of the firearm. The fore-end may in some cases include aportion of the action structure of the firearm. For example, thefore-end may include a pump component for a pump action shotgun or otherpump action firearm. The stock structure may also include a triggerunit, which includes a trigger engaged by a user of the firearm and mayalso include a safety for selectively disengaging the operation of thetrigger. The stock structure may also include a magazine well whichreceives a magazine and directs a projectile from a cartridge insertedin the magazine to a chamber of the barrel structure. In embodiments,the trigger unit and/or the magazine well may be included in the stockstructure. In embodiments, the grip may be included in a portion of thestock structure other than the butt. In embodiments, the grip may beincluded in a component in contact with the stock structure instead ofin the stock structure itself.

The barrel structure is or refers to a structure of components throughwhich a projectile is fired, for example, using combustion orcompression. In embodiments, the barrel structure includes a chamber, amuzzle, and a bore. The chamber is a cavity in which an ammunitioncartridge is inserted and in which a projectile is stored until it isfired. The muzzle is the portion of the barrel structure through which aprojectile is fired, and which is located at an end of the barrelstructure opposite to the chamber. The muzzle may, in embodiments,include a coupling element, which may, for example, be or include athreaded engagement or another engagement. An accessory device for usewith the firearm may be coupled to the coupling element on the muzzle oranother portion of the barrel structure. For example, the accessorydevice may be coupled by a coupling element located above the muzzlewhen the firearm is oriented for normal operation. In such a case, forexample, the accessory device may be a sight, a scope, or anotheraccessory. In another example, the accessory device may be coupled by acoupling element located in front of the muzzle when the firearm isoriented for normal operation. In such a case, for example, theaccessory device may be a flash hider, a suppressor, or anotheraccessory. The bore is the hollow length of the barrel structure throughwhich a projectile travels when fired. An internal surface of the boremay, in embodiments, be smooth or grooved to control or otherwise enablea projection of a projectile from the chamber to a location outside ofthe muzzle during firing.

A rail is or refers to a structure to which one or more accessories maybe coupled for use during the operation of the firearm. A rail includesan interface mechanism for permanently or removably coupling accessoriesto the firearm. The interface mechanism may allow for one or more ofslidable engagement of an accessory, slotted engagement of an accessory,threaded engagement of an accessory, snap-fit engagement of anaccessory, friction-fit engagement of an accessory, or the like. Therail may be a Dovetail rail, a Weaver rail, a Warsaw Pact rail, aPicatinny rail, a KeyMod rail, a M-LOK rail, or a UIT rail, althoughother styles of rail are possible. In embodiments, the particular formof the interface mechanism may depend upon the style of the rail. A railas used with a firearm according to the embodiments of this disclosuremay be coupled to a surface of an action structure of a firearm (e.g.,above an ejection port), a surface of a barrel structure of a firearm(e.g., above the chamber or a portion of the muzzle), or a surface of astock structure of a firearm (e.g., above a handguard). Although a railtypically is located on an upper surface of a firearm structure withrespect to an orientation of the firearm during use, in embodiments, arail as disclosed herein may be located on another surface, or on acombination of surfaces, of one or more firearm structures. Examples ofaccessories which may be coupled to a rail include, without limitation,scopes, sights (e.g., laser sights, iron sights, reflector sights,holographic sights, or the like), tactical lights, and vertical forwardgrips.

In embodiments, components described above as being included in theaction structure, as being included in the stock structure or being incontact with the stock structure, or as being included in the barrelstructure, may instead be included in one of a lower receiver unit ofthe firearm or an upper receiver unit of the firearm. In embodiments,components described herein as being included in the stock structure mayinstead be included in the lower receiver unit and/or the upper receiverunit, or both. In embodiments, one or more rails and/or componentscoupled to rails as described above may be included in the lowerreceiver unit and/or the upper receiver unit.

Firearm 200 includes action structure 204, stock structure 206, andbarrel structure 208. Action structure 204 is shown as includingcharging handle, bolt, and ejection port. Stock structure 206 is shownas including grip, comb, handguard, trigger unit, magazine well, andmagazine. Barrel structure 208 is shown as including muzzle, accessorydevice (e.g., a suppressor), and accessory device (e.g., a sightassembly). Firearm 200 further includes first rail 210 and second rail212. Each of the rails 210 and 212 includes an interface mechanism forpermanently or removably coupling one or more accessories to firearm200. For example, first accessory 214 (e.g., a laser sight and/ortactical light) is coupled to rail 212 and second accessory 216 (e.g., ascope) is coupled to rail 210. In embodiments, other components and/orother numbers of components may be coupled to rail 210 and/or to rail212. In embodiments, action structure 204, stock structure 206, andbarrel structure 208 may include components other than or in addition towhat is shown in FIG. 2 .

In embodiments, a firearm used in connection with a firearm monitoringand remote support system in accordance with the embodiments of thepresent disclosure can include structures other than an actionstructure, a stock structure, a barrel structure, and/or one or morerails. For example, in embodiments, such a firearm can include acylinder structure including multiple chambers for storing a projectileto be fired. For example, the firearm may be a revolver or anotherfirearm with a structure for rotating multiple chambers into alignmentwith the bore of the barrel structure. In another example, inembodiments, such a firearm may omit the stock structure. For example,the firearm may be a pistol or other handgun in which components such asthe grip and/or trigger are coupled to the rest of the firearm by astructure other than a stock structure. In another example, inembodiments, such a firearm may include a stock structure that omits thebutt. For example, the firearm may be a pistol or other handgun whichincludes a stock structure that structurally supports the actionstructure and/or the barrel structure, but in which contact with theuser is intended to be limited to the grip. It is to be understood thatother firearm embodiments as are currently known or which are laterdeveloped may be used to implement or otherwise integrate one or more ofthe methods and systems disclosed herein.

Assets used in connection with a firearm monitoring and remote supportsystem in accordance with the embodiments of the present disclosure maybe located within or otherwise positioned with respect to certainstructures and/or certain components of structures used in connectionwith firearm 200. Examples of such structures are shown in FIG. 2 aswearable devices worn by user 202 of firearm 200. The examples includeouterwear 218, helmet 220, earpiece 222, eyeglasses 224, and wristbands226. Outerwear 218 may be or include a vest, a jacket, a shirt, oranother wearable item. Helmet 220 may be a helmet or another headcovering or combination of head coverings. Earpiece 222 is an in-eardevice for receiving audio from a remote source. In embodiments,earpiece 222 may include a microphone for recording audio fortransmission to another in-ear device or to a remote source. Inembodiments, earpiece 222 may be a hearing guard, such as a plug forblocking the ear canal of user 202. In such an embodiment, earpiece 222may omit audio communication functionality. Eyeglasses 224 are a coverfor one or both eyes of user 202. Wristbands 226 are wearable devicesworn around the wrists of user 202. Although one wristband 226 is shownon each arm of user 202, in embodiments, user 202 may wear a wristband226 on only one arm, or user 202 may wear more than one wristbands 226on one or both arms. In embodiments, one or more of outerwear 218,helmet 220, earpiece 222, eyeglasses 224, or wristbands 226 may beembodied in a form factor other than what is shown as described. Forexample, one or both of wristbands 226 may be embodied as rings worn onfingers of user 202, as devices worn around a neck of user 202, as pinscoupled to outerwear 218, or another form factor, or a combinationthereof. In embodiments, outerwear 218 may be or include clothing orother wearable items which are not located worn as outerwear. Forexample, outerwear 218 may be or include an undershirt, a vest wornunderneath outerwear, or another wearable item.

In embodiments, assets other than wearable devices used in connectionwith a firearm monitoring and remote support system in accordance withthe embodiments of the present disclosure may be located within orotherwise positioned with respect to certain structures and/or certaincomponents of structures used in connection with firearm 200. Althoughnot shown in FIG. 2 , examples of such other assets include mobiledevices (e.g., cell phones, tablet computers, personal digitalassistants (PDAs), mobile connection points, or the like) which may bepossessed by the user and/or permanently or removably coupled to otherassets (e.g., firearms, wearable devices, stationary devices, stationaryconnection points, or the like).

While examples of particular structures of a firearm and particularcomponents of structures of a firearm are disclosed herein, suchdisclosure is not limiting as to the possible structures of componentsof structures of a firearm or as to the possible locations orpositionings of components used by the methods and systems disclosedherein with respect to those structures or those components ofstructures. Accordingly, it is to be understood that components used byone or more of the methods and systems disclosed herein may be locatedor positioned in other locations or positions in or about a firearm,regardless of the particular structures disclosed herein by example.

Referring to FIG. 3 , computing device 300 is or refers to one or moreof: server device 112; an electronic system within or otherwise coupledto a firearm 104, a wearable device 106, a stationary device 108, orresponse infrastructure 110; or another computer, phone, PDA, or othersort of electronic device used in connection with system 100.

Computing device 300 includes various types of computer readable mediaand interfaces for various other types of computer readable media.Computing device 300 includes bus 302, processing unit(s) 304, systemmemory 306, read-only memory (ROM) 308, permanent storage device 310,input devices 312, output devices 314, and network interface 316.

Bus 302 collectively represents all system, peripheral, and chipsetbuses that communicatively connect the numerous internal devices of thecomputing device 300. For instance, bus 302 communicatively connectsprocessing unit(s) 304 with ROM 308, system memory 306, and permanentstorage device 310. From these various memory units, processing unit(s)304 retrieves instructions to execute and data to process in order toexecute the many processes disclosed herein. The Processing unit(s) 304may be or include a single processor or a multi-core processor indifferent embodiments. In embodiments, the system memory 306 could alsobe used as a buffer for data before the data is transmitted from theuser. In embodiments, the system memory 306 could also be used as abuffer for data before being sent to storage, especially in situationswhere the data cannot be transmitted from the user.

ROM 308 stores static data and instructions that are needed byprocessing unit(s) 304 and other modules of computing device 300.Permanent storage device 310, on the other hand, is a read-and-writememory device. The Permanent storage device 310 is a nonvolatile memoryunit that stores instructions and data even when computing device 300 isoff. Some embodiments disclosed herein may use a mass-storage device(such as a magnetic or optical disk and its corresponding disk drive) aspermanent storage device 310.

Other embodiments use a removable storage device (such as a floppy diskor a flash drive) as permanent storage device 310. Like permanentstorage device 310, system memory 306 is a read-and-write memory device.However, unlike storage device 310, system memory 306 is a volatileread-and-write memory, such as random access memory (RAM). System memory306 stores some of the instructions and data that the processor needs atruntime. In some embodiments, processes are stored in system memory 306,permanent storage device 310, and/or ROM 308. For example, the variousmemory units include instructions for processing appearance alterationsof displayable characters in accordance with some embodiments. Fromthese various memory units, processing unit(s) 304 retrievesinstructions to execute and data to process in order to execute thevarious processes of disclosed herein.

Bus 302 also connects to input devices 312 and output devices 314. Inputdevices 312 enable the person to communicate information and selectcommands to computing device 300. Input devices 312 include alphanumerickeyboards and pointing devices (also called cursor control devices).Output devices 314 display images generated by computing device 300.Output devices 314 include printers and display devices, such as cathoderay tubes (CRTs), liquid crystal displays (LCDs), or light-emittingdiodes (LEDs). Some embodiments include devices such as a touchscreenthat functions as both input devices 312 and output devices 314.

Bus 302 also couples computing device 300 to network interface 316 forconnecting computing device 300 to a network (e.g., network 114). Inthis manner, the computing device 300 can be a part of a network ofcomputers (e.g., a LAN, a WAN, a VPN, a P2P network, or an intranet), anetwork of networks (e.g., the Internet), or another network (e.g., acellular network). Any or all components of computing device 300 may beused in conjunction with the various embodiments of the presentdisclosure. For example, network interface 316 can enable communicationsover Ethernet, TCP, IP, power line communication, Wi-Fi, Bluetooth®,infrared, RF, GPRS, GSM, FDMA, CDMA, EVDO, Z-Wave, ZigBee, 3G, 4G, 5G,another protocol, or a combination thereof.

Referring to FIG. 4 , the functionality of application 102 is furtherdescribed. Application 102 includes software modules used for monitoringfirearms and other assets within a deployment location (e.g., wearabledevices and/or stationary devices). The software modules include firearmmonitoring module 400, threat detection and analysis module 402, threatresponse module 404, GUI generation and display module 406, and signalprocessing module 408.

Firearm monitoring module 400 monitors firearms (e.g., firearms 104),deployed to a deployment location. Firearm monitoring module 400monitors the users, firearms, and other assets based on measurementsrecorded using sensors included within or otherwise coupled to thefirearms and other assets (e.g., sensors 118). For example, the firearmmonitoring module 400 can process the measurements recorded using thesensors to determine changes in a position and/or orientation of afirearm and/or to determine motion of the firearm. For example, themeasurements recorded using the sensors may include or otherwise beindicative of one or more of a change in orientation of a firearm, avibration of the firearm, recoil resulting from a firing of the firearm,pressure applied to all or a portion of the firearm (e.g., to a triggermechanism or grip), changes in contents of a magazine of the firearm,heat and/or light changes at a muzzle of the firearm (e.g., indicating afiring of the firearm), or the like.

In embodiments, firearm monitoring module 400 monitors users of firearms(e.g., users of firearms 104) and/or other assets (e.g., wearabledevices 106 and stationary devices 108). Firearm monitoring module 400monitors the users and/or other assets based on measurements recordedusing sensors included within or otherwise coupled to the other assets(e.g., sensors 122) and sensors included in assets worn by the users(e.g., sensors 120). For example, the firearm monitoring module 400 canprocess the measurements recorded using the sensors to determine changesin a position and/or orientation of a user and/or other asset and/or todetermine motion of the user and/or other assets. For example, themeasurements recorded using the sensors may include or otherwise beindicative of one or more of a sudden motion of the user, a speed and/ordirection of motion of the user, a vibration measured based on a firingof a firearm of the user, sudden changes in an amount of light detectedaround the user, or the like.

The output of firearm monitoring module 400 can be used to updateinformation representing real-time position and orientation of firearmsin a deployment location. For example, as will be described below, theoutput of firearm monitoring module 400 can be used by GUI generationdisplay module 406 to update one or more GUIs to change a visualrepresentation of one or more firearms based on the real-time positionand orientation of the firearms. In another example, as will bedescribed below, the output of firearm monitoring module 400 can be usedby threat detection and analysis module 402 to detect a threat withinthe deployment location and/or to analyze whether a threat detectedwithin the deployment location requires a threat response.

In embodiments, firearm monitoring module 400 includes functionality formonitoring firearm maintenance. With such firearm maintenancemonitoring, the application 102 may provide (e.g., to the user of afirearm, to remote support personnel using the application 102, or toother personnel) data on the number of rounds discharged and whichfirearm components need maintenance or replacement. The firearmmaintenance monitoring functionality of firearm monitoring module 400can include generating an alert indicative of maintenance requirementsdetermined based on the monitoring, for example, to notify the user ofthe firearm, remote support personnel, or other personnel.

In embodiments, firearm monitoring module 400 includes functionality foralerting a user of a firearm should the firearm is pointed at anotheruser (e.g., in the same group or otherwise). For example, each user mayhave a tracking system (e.g., included in the firearm, a wearable deviceworn by the user, or another asset). The firearm monitoring module 400can detect when a firearm of one user is pointed at another user withsuch a tracking system. In some such embodiments, firearm monitoringmodule 400 may also alert the user should the firearm be pointed atother weaponry (e.g., another firearm or another weapon), anotherdeployed asset, another predefined target, raised quickly in ageo-defined zone, or the like. This may, for example, help avoidfriendly fire (e.g., potentially resulting in fratricide) situations.

Firearm monitoring module 400 includes functionality for identifyingdischarges and counting shots, discharges, and other operations offirearms. In embodiments, an external device attached to a firearm canregister when a shot is fired. The discharge has a unique, detectable,physical profile (e.g., a discharge has recoil that has a particularmotion profile, sound profile, and the like). A recoil measuring systemmay use an IMU, including or combined with motion-detecting/sensingelements, including one or more accelerometers, gyros, magnetometers,and the like. In embodiments, a map is developed based on analyses ofdischarge events to the map the entire motion sequence caused by atypical discharge. That motion profile, which may be unique to eachweapon platform and user, can be stored and used as a basis forcomparing future sensed data to determine whether a discharge event hasoccurred. Similar profiling can be used for each weapon type todetermine whether the firearm has been raised to an aiming position orout of the holster position.

In embodiments, a firearm may include an infrared gate in front of theejection port. This gate can track disconnects when the weapon is fired,such as when the shell is engaged and breaks the gate. In embodiments, afirearm may include a hall-effect sensor to measure the motion of aninternal part. In embodiments, firearm monitoring module 400 can capturethe discharge profile of a given weapon by using an IMU. The dischargeprofile may have unique inertial characteristics when a weapon isdischarged, such as based on the geometry, distribution of weight,specified ammunition, and the like, so that a discharge can be profiledand identified based on a series of movements that are measured by theIMU. In embodiments, firearm monitoring module 400 can capture thedischarge profile of a given weapon by using a sensor to monitor theposition of the trigger so that when the trigger is pulled, we canassume a discharge and verify with the IMU or other correlating data oruse that to identify misfire or dry fire scenario.

In embodiments, firearm monitoring module 400 includes an activitymonitor which will indicate events such as when the gun is elevated andbeing pointed. In embodiments, firearm monitoring module 400 measuresthe parameters of the recoil and parameters of pre-shot movement. Thisallows an analysis of changes over time to determine the status of theweapon. The firearm monitoring module 400 can also capture movements anddetermine whether the user is handling the weapon properly.

In embodiments, firearm monitoring module 400 provides alternatives formonitoring discharges, such as cameras, or augments those othermonitoring systems. The methods and systems disclosed herein may includeimage recognition, which can identify the flash of a muzzle or for theslide rocking back. The system may also have acoustic abilities and mayprovide sound recognition.

Threat detection and analysis module 402 uses sensor measurements,existing knowledge, and/or trained machine learning models related tousers, firearms, other assets, and/or conditions within a deploymentlocation to detect whether a threat exists within the deploymentlocation and to analyze the severity of detected threats. The sensormeasurements, existing knowledge, and/or trained machine learning modelsare taken as inputs to threat detection and analysis module 402. Threatdetection and analysis module 402 outputs data indicating one or more ofa detected threat, a threat severity, or no threat detected. The sensormeasurements can include measurements recorded using one or more ofsensors 118, sensors 120, or sensors 122. For example, measurementsrecorded using one or more of sensors 118, sensors 120, or sensors 122may include, without limitation, information indicating changes in theorientation of a firearm, changes in a movement speed of a user,biometric information (e.g., increase in user pulse, increase in usersweat levels, increase in user eye movement, or the like), changes inhow a user grips or otherwise holds a firearm, image or video data(e.g., captured using a user body camera, a camera on a UAV or otheraircraft, a camera of a stationary device, a camera of another vehicle,or another camera) showing hostiles or persons who appear to behostiles, or the like.

In embodiments, the sensor measurements can include information outputfrom firearm monitoring module 400. For example, the firearm monitoringmodule 400 can process measurements recorded using one or more ofsensors 118, sensors 120, or sensors 122 and provide the processedmeasurements to threat detection and analysis module 402. In such anembodiment, the processed measurements may include additionalinformation added by firearm monitoring module 400, for example,representing previous monitoring data corresponding to one or more ofsensors 118, sensors 120, or sensors 122.

The knowledge used by threat detection and analysis module 402 includesinformation stored within a knowledgebase of system 100. For example,the knowledgebase can be represented by or otherwise include or refer toinformation stored within database 124, which can be accessed byapplication 102. The information within the knowledgebase can beprocessed against the sensor measurements, for example, to compare thesensor measurements to established thresholds or other known conditionsassociated with threat detection. As will be described below, themachine learning models used by threat detection and analysis module 402are models (e.g., of a deep learning neural network or another machinelearning or machine intelligence approach) which have been trained usingone or more data sets (e.g., including information within theknowledgebase and/or information collected from past deployments ofusers using sensors such as sensors 118, sensors 120, and/or sensors122).

Threat detection and analysis module 402 can use a rule-based approachto detect and analyze threats. In embodiments, different inputs may beassigned or otherwise attributed different score values. For example, afirearm discharge being detected may have a higher score than a changein firearm orientation detection. In another example, a change infirearm orientation detection may have a higher score than a user motiondetection. A rule used by threat detection and analysis module 402 cancompare score totals calculating by adding scores for various presentinputs against one or more thresholds configured for indicating athreat. A threat can be detected where the calculated score total meetsor exceeds one or more of the thresholds. In embodiments, a rule used bythreat detection and analysis module 402 can indicate that the detectionof a condition (e.g., based on one or more of the sensor measurements)indicates a detected threat. For example, where the sensor measurementsindicate that multiple firearms are being discharged at the same time,threat detection and analysis module 402 can use that information aloneto determine that a potential threat exists.

Threat detection and analysis module 402 analyzes the input informationnot only to detect a threat, but also to determine a severity of adetected threat. For example, a detected threat which appears to relateto the presence of a single hostile combatant may be considered lesssevere than a detected threat which appears to relate to the presence ofmultiple hostile combatants. In another example, a detected threat whichappears to relate to the presence of enemy tanks, mortars, or otherheavy machinery or heavy-powered weaponry may be considered more severethan a detected threat which appears to relate to the presence ofhostile combatants armed only with handguns or assault rifles. Thedetection of a threat, along with the severity of the detected threat,are used to determine an appropriate response to the detected threat,for example, using threat response module 404. Determining the severityof a threat may include analyzing some or all of the input informationused to detect the treat. For example, for a given threat, a threatseverity may be low when input information indicates that the users whowill engage the threat have what is expected to be an adequate amount ofammunition for engaging the threat, but high when the input informationindicates that those users do not have an adequate amount of ammunition.In another example, for a given threat, a threat severity may be lowwhen the number of users who will engage the threat is greater than thenumber of hostile combatants detected, but high when the number of userswho will engage the threat is less than the number of hostile combatantsdetected.

Threat detection and analysis module 402 thus considers the types andnumber of firearms possessed by each engaged user, the amount of unusedammunition remaining in possession of each such engaged user, the amountof ammunition already used during the engagement by each such user, thenumber of engaged users, the locations of the engaged users (e.g., inrelation to each other, to the hostiles, and/or to the geography), andthe like. Threat detection and analysis module 402 further considers thenumber of hostiles, the locations of the hostiles (e.g., in relation toeach other, to the users, and/or to the geography), the types offirearms used by the hostiles (including the expected ammunition stocksand possible reserves therefor), the number of firearms used by thehostiles, an amount of time for which the firearms of the users havebeen firing at the users, and the like.

In embodiments, system 100 can prompt a user of a firearm for inputverifying that a threat exists. For example, where threat detection andanalysis module 402 detects a potential threat with low confidence, asignal may be transmitted to a personal computing device of a userwithin the deployment location to verify whether a threat is present inthe deployment location. The user may respond in one or more ways toverify the potential threat. For example, the personal computing devicemay include a button or other hardware interface which may be toggled inresponse to the request for verification to indicate whether the threatexists. In another example, the personal computing device or a wearabledevice may include a microphone. The user can speak into the microphoneto verify whether the threat exists. Other options for verifying athreat are possible, as will be understood. In embodiments, a mode ofthe safety on the firearm can be detected and can be a further optionfor verifying a threat such that one of the modes (e.g., safe, semi,full-auto, locked) can be an additional metric when assessing the threatdetection.

In embodiments, system 100 may allow a user (e.g., of a firearm and/orof a remote dashboard) to validate a threat using the firearm, forexample, during in a live combat situation. For example, application 102may establish or otherwise be used to establish a pressure signature tovalidate the threat. The threat may be validated by application 102(e.g., by threat detection and analysis module 402) by comparing thepressure signature against a range of pressure signatures, for examplefrom no pressure to extreme pressure.

The pressure signature may be established by collecting information,such as information from sensors, for example, sensors 118, sensors 120,sensors 122, and/or other sensors, such as multi-modal sensors 1060.Combinations of sensors may include combinations of wearable and firearmsensors, combinations of the firearm and fixed sensors, for example,Internet of Things (IoT) sensors, and the like. A sensor equippedfirearm may include a pressure sensor, for example, to determine a gripprofile using information such as threat ID, shot accuracy, engagement,alert information and tactical information. Information collected from asensor equipped firearm may include discharge information, motioninformation, rate of motion information, orientation information and thelike. The rate of motion information, for example, may include movementinformation related to speed, threat identification and shot accuracy.Movement information may also be related to an event identifier forevents, such as events associated with weapons and people. Eventsassociated with firearms may include events indicating the firearm hasfallen, is outside of a pre-designated distance from its owner, in anunauthorized area and the like. Events associated with people mayinclude events indicating a person is in an unauthorized area, themaneuvering speed of the person and the like.

In embodiments, determining the pressure signature may also includedetermining a firearm-specific candidate action of a first firearm user,from at least a portion of the collected information. The candidateaction may be compared with other firearm users, for example, otherfirearm users proximal to the first firearm user or other firearm usersassociated with the first firearm user. The collected information,candidate action or actions, and action comparison result may then bestored in a data structure that represents the pressure signature. Thecollected information, candidate action or actions, and actioncomparison result may also be filtered or weighted based on specifiedcriteria, prior to being stored in the data structure that representsthe pressure signature.

Threat response module 404 determines an action to be performed inresponse to a detected threat based on the detected threat and based onthe severity of the detected threat. The action determined using threatresponse module 404 may be based on the threat detected. As such, thethreat response module 404 can use information, qualities,characteristics, or other aspects of the detected threat (e.g.,identified, determined, or otherwise produced using threat detection andanalysis module 402) to determine the action to perform in response tothe detected threat. Examples of actions which may be determined usingthreat response module 404 include, but are not limited to, delivery ofadditional ammunition to one or more users within the deploymentlocation, request for reinforcements within the deployment location toassist in engaging the threat, delivery of new firearms or otherweaponry (e.g., weaponry which is heavier or otherwise more powerfulthan is currently possessed by the users within the deployment location)to one or more users within the deployment location, delivery of medicalequipment to one or more users within the deployment location, requestfor medical personnel within the deployment location (e.g., with orwithout medical equipment), delivery of new communications tools withinthe deployment location, transmissions of notifications to nearbyconnection points (e.g., to notify another group of users as to theexistence of the threat detection or engagement), or the like.

The threat response module 404 can use a rule-based approach todetermine an appropriate threat response. In embodiments, the rule-basedapproach used by threat response module 404 may be the same rule-basedapproach as may be used by threat detection and analysis module 402. Inembodiments, the rule-based approach used by threat response module 404may be an extension of the rule-based approach used by threat detectionand analysis module 402. The rule-based approach used by threat responsemodule 404 can indicate to determine certain threat responses based oncertain detected threats and/or based on certain severities of detectedthreats. For example, a rule can indicate to deliver additionalammunition (e.g., by UAV or otherwise) when a detected threat includesmultiple hostile combatants and the associated threat severity is high.In another example, a rule can indicate to request reinforcements toarrive at the deployment location within some specified or unspecifiedamount of time when a detected threat includes a number of hostilecombatants which is higher than a number of engaging users and the skilllevels of the engaging users do not meet a threshold.

In embodiments, the action determined using threat response module 404can include or otherwise indicate a combination of actions to beperformed in response to a detected threat. For example, where threatdetection and analysis module 402 determines that a given user will runout of ammunition before the end of an engagement with a number ofhostiles and that the number of hostiles exceeds the number of users inthe group that includes the given user, threat response module 404 candetermine the action to be performed in response to the detected threatas delivering additional ammunition to the given user (e.g., by UAVdelivery or otherwise) and calling for reinforcements to assist thegroup of users in engaging the number of hostiles.

GUI generation and display module 406 generates, updates, and renders ordisplays GUIs. A GUI generated using GUI generation and display module406 can comprise part of a software GUI constituting data that reflectinformation ultimately destined for display on a hardware device, forexample, a client device or other computing device which communicateswith server device 112 or another computing device running, executing,interpreting, or otherwise operating application 102. For example, thedata can contain rendering instructions for bounded graphical displayregions, such as windows, or pixel information representative ofcontrols, such as buttons and drop-down menus. The renderinginstructions can, for example, be in the form of HTML, SGML, JavaScript,Jelly, AngularJS, or other text or binary instructions for generating aGUI or another GUI on a display that can be used to generate pixelinformation. A structured data output of one device can be provided toan input of the hardware display so that the elements provided on thehardware display screen represent the underlying structure of the outputdata. Instructions for displaying or otherwise rendering a GUI generatedusing GUI generation and display module 406 can be communicated fromserver device 112 to a client device or another computing device whichcommunicates with server device 112.

GUIs which may be generated, updated, and rendered or displayed usingGUI generation and display module 406 include a deployment location GUI,a remote support dashboard GUI, a user and firearm GUI, and others. Thedeployment location GUI includes a two-dimensional top-down geographicview of the deployment location including icons indicating positions andorientations of users and/or of firearms or other assets within thedeployment location and further including cones for the users, firearms,or other assets. The top-down geographic view may, for example,represent a real-time satellite feed imaging the deployment location.Alternatively, the top-down geographic view may represent terrain,topographic, roadway, or other map views of the deployment location.

The remote support dashboard GUI includes views for displaying andenabling user interaction with information relating to users, firearms,and/or other assets deployed to the deployment region. For example, theremote support dashboard GUI may include a dashboard view which displaysone or more of lists of users, lists of firearms possessed by the users,stock of ammunition possessed by the users, lists of potential or actualthreats detected within the deployment location, alerts corresponding todetected threats, alerts corresponding to detections of firearms beingfired, or the like. In another example, the remote support dashboardview may include a validation view which presents requests, actions, orother information for review and/or approval by a user of application102. The validation view may, for example, display notificationsrelating to automated responses taken based on detected threats. Inembodiments, where a response is presented for user approval beforeexecution, the validation view may present a request to approve aresponse.

The user and firearm GUI include views indicating real-time position andorientation information for users and firearms used thereby. Forexample, the user and firearm GUI may include a three-dimensionalfirearm orientation view which updates in real-time based on signalsreceived from a firearm to show an orientation of the firearm, forexample, with respect to a surface on which a user of the firearm isstanding or otherwise located. In another example, the user and firearmGUI may include a two-dimensional recoil tracking view which updates inreal-time based on signals received from a firearm to show how thefirearm moves over time based on recoil from firings of the firearm. Inyet another example, the user and firearm GUI may include a view showingreal-time video or image feeds captured using a body camera of a user.

In embodiments, two or more GUIs, or views from two or more GUIs, may becombined into a single GUI which is rendered or displayed. For example,some or all views of the user and firearm GUI may be included in thedeployment location GUI, for example, to enable the simultaneous displayof multiple monitors. For example, as will be described below withrespect to FIG. 6 , a deployment location GUI may include a top-downgeographic view, a three-dimensional firearm orientation view, atwo-dimensional recoil tracking view, and a user body camera feed view.In this way, a remote user of application 102 can simultaneously viewreal-time information regarding user hostile engagement or detectionwithin the deployment location and individual or group firearmmonitoring information.

Signal processing module 408 processes signals received, directly orindirectly, from assets within a deployment location. The signalprocessing module 408 can receive and process signals from firearms 104,wearable devices 106, stationary devices 108, and/or other assets.Processing signals using signal processing module 408 includes preparingdata included within those signals for use with other modules ofapplication 102. For example, signal processing module 408 can process asignal to prepare the signal for use by one or more of firearmmonitoring module 400, threat detection and analysis module 402, or GUIgeneration and display module 406. For example, a signal received atapplication 102 may be received in a compressed form. The signalprocessing module 108 processes the signal including by decompressingthe signal to restore the data included in the signal to an uncompressedform. In another example, a signal received at application 102 mayinclude noise, for example, introduced during the recording of sensormeasurements (e.g., by motion of a user, vibrations to which a sensor isexposed, or another noise source). The signal processing module 408 candenoise the signal before making the signal available to one or more offirearm monitoring module 400, threat detection and analysis module 402,or GUI generation and display module 406.

In embodiments, signal processing module 408 can receive and processbatches of signals. For example, rather than receiving a sequence ofindividual signals, signal processing module 408 can receive a batch ofsignals generated, identified, or otherwise collected (e.g., usingconnection point 116) for transmission to server device 112 for use withapplication 102. For example, a batch of signals may represent signalscollected within a defined time interval (e.g., within a five secondperiod or less). For example, connection point 116, or another componentwhich collects signals from assets deployed within a deploymentlocation, can use timestamps for the signals to coordinate batching ofsignals for transmission to server device 112. In another example, abatch of signals may represent signals relating to common asset types orfrom a specific asset or group of assets. For example, connection point116, or another component which collects signals from assets deployedwithin a deployment location, can determine the type of asset from whicha signal is collected (e.g., based on pre-processing performed againstthe signal and/or based on a channel of communication used to collectthe signal) and can coordinate batching of signals for transmission toserver device 112 by grouping the signals by asset type.

In embodiments, application 102 includes inventory controlfunctionality. For example, the inventory control functionality caninclude monitoring stores of asset inventory (e.g., firearms,ammunition, wearable devices, stationary devices, and/or other assets)within one or more locations. The inventory control functionality can beused to track when assets are taken out of an inventory store (e.g., foruse in arming a user during a deployment). The inventory controlfunctionality can also be used to track inventory usage, for example, toassist in determining when resupply orders are needed. In some suchembodiments, the inventory control functionality of application 102 caninclude functionality for automating resupply orders of some or allasset inventories, for example, based on the monitoring of the assetswithin one or more locations of the inventory stores and/or within oneor more deployment locations.

In embodiments, application 102 includes predictive functionality. Insome such embodiments, the predictive functionality of application 102can include functionality for determining an action to be performed evenin the absence of a detected threat. For example, the predictivefunctionality of application 102 can include a predictive resupplymodule that predicts a need to resupply ammunition based on the numberof shots taken using one or more firearms. The predictive functionalityof application 102 can include generating an alert indicative of theaction to be performed, for example, to notify a user of the firearm,remote support personnel, or other personnel. In embodiments, whichinclude such predictive resupply module and inventory controlfunctionality, the inventory control functionality can account forinventory of rounds used with the predictive resupply module that tracksthe amount of ammunition used and alerts when the inventory and shotsfired do not match indicating a loss of ammunition.

In other such embodiments, the predictive functionality of application102 can include functionality for predicting maintenance or other statesof assets within a deployment location. For example, the predictivemaintenance can include predicting a maintenance requirement and/orstatus of a firearm based on a number of shots taken, based on recoilparameters (e.g., showing degradation of performance as recoil patternsshift over time), and/or based on other criteria. The predictivefunctionality of application 102 can include generating an alertindicative of the predicted maintenance requirement and/or status, forexample, to notify a user of the firearm, remote support personnel, orother personnel.

Beneficially, the firearm usage monitoring system may providemaintenance alerts and confirmation of maintenance performed on afirearm without user input. In embodiments, the firearm usage monitoringsystem is configured to monitor round count and fatigue (e.g., heat fluxand temperature buildup from discharge events) to determine whenreplacement of consumable or degradable components is likely.Beneficially, the firearm usage monitoring system may include supplychain information (e.g., deployed inventories or inventories at depot,resupply, or global resupply) to alert a resupply need or automaticallyresupply components. In embodiments, sensors on the firearm 104 areconfigured to monitor the noise, vibration, and harshness signature(NVH) to determine potential failure modes (e.g., NVH increaseindicative of overheating event) and/or maintenance (e.g., NVH decreaseindicative of component replacement or cleaning).

In embodiments, application 102 can use machine learning functionality(e.g., implemented as one or more machine learning modules ofapplication 102) for training and/or inference. For example, the machinelearning functionality of application 102 can be used to trainapplication 102 based on information input to or output from one or moreof modules 400-408. In another example, the machine learningfunctionality of application 102 can perform inference againstinformation input to or output from one or more of modules 400-408. Inyet another example, the machine learning functionality of application102 can perform both the training and the inference described above.

In embodiments, the machine learning functionality of application 102can include algorithms for determining recoil of firearms 104 and otherbehaviors or characteristics of system 100. For example, in embodiments,the machine learning functionality of application 102 includesidentification algorithms to determine the complex motion associatedwith the discharge of a particular type of weapon. Embodiments mayinclude feeding IMU data collected upon gripping, movement, anddischarge of weapons into the machine learning functionality ofapplication 102, for example, so that the machine learning functionalityof application 102 can learn the parameters of each with respect toenough training events that it can rapidly and accurately identify newevents based on new IMU data, such as collected in real time. Inembodiments, the machine learning functionality of application 102 canbe trained to learn to identify a threatening situation when the grip isengaged and the firearm is pointed, when the motion has increasedindicating a pursuit, and when it is not in motion (e.g., placed insleep mode). More complex patterns can be learned, such as determiningwhat patterns tend to lead to accidents, dangerous incidents, higherquality training, and the like.

In an example of learning and utilization of a complex pattern, themachine learning functionality of application 102 can be used todetermine firearm movements that may indicate a discharge from a firearmis imminent. In this example, the machine learning functionality ofapplication 102 may, for example, detect motion and orientation datafrom sensors, such as from sensors on the firearm, sensors in a meshnetwork (e.g., including other firearms), or other assets (e.g., sensorswithin wearable devices, multi-modal sensors, etc.) of the human user ofthe firearm, which in turn may be used by the machine learningfunctionality of application 102 to facilitate a threat response. Inembodiments, a threat response may include an automatic threat response,such as by one or more machines that are teamed with the human user ofthe firearm.

In another example of learning and utilization of a complex pattern, themachine learning functionality of application 102 can considerinformation stored within a knowledgebase or other data store (e.g., ofdatabase 124 or another source). For example, the information may relateto past engagements of users, whether or not involving the same users asare currently deployed within a given deployment location. Theinformation may, for example, relate to one or more of a user skilllevel, firearm type, amount of ammunition used in engagements based onuser skill level and/or firearm type, numbers of engagements of users,numbers of threats or otherwise of hostile combatants or weaponryengaged, number of users in a group which engaged a threat, number offirearms possessed per user of such a group, or the like.

In embodiments, the machine learning functionality of application 102may determine combinations of data, such as motion, orientation andmulti-modal sensor information that are indicative of imminent dischargeof the firearm. The machine learning functionality of application 102may also receive other inputs or generate information to combine withthe sensor information, such as an indication of a firearm state.Firearm states may include combat states, training states, wartimestates, peacetime states, civilian states, military states, firstresponder states, incident response states, emergency states, militarycontractor states, on-call states, and the like. Firearm states may bestates from one or more than one firearm, for example, a set of firearmsassociated with a group of soldiers in the same section of a battlefieldor a set of police officers in a region.

Combinations of data may allow the machine learning system to recognize,determine, classify, or predict information, such as about environments,objects, image content, whether a person is friendly or adversary,structures, landscapes, human and human gestures, facial indicators,voices, and locations, among others. Example combinations may includecombinations of data from topography and physiological monitors, ISR,and structure recognition combinations, as well as combinations of humanand machine physical states. Combinations of data may also be tacticalcombinations. Tactical combinations may combine data from devices on abattlefield, information about other sectors of fire, and the like andmay include firearms and other weapons, vehicles, body armor and otherwearable elements, and the like (collectively referred to herein as“battlefield of things”) devices including, for example, remotelyoperated units such as Common Remotely Operated Weapon Stations (CROWS)or other remote controlled firearms that may be configured with heaviercalibers and higher lethality.

Objects that may be recognized by machine learning may include weapons,man-made objects, natural objects, and the like. Structures may includedoors, stairs, walls, drop-offs, and the like. Human gestures may bedetected, interpreted and understood by the machine learning system,while facial indicators could be indicators of mood, intent, and thelike. The machine learning functionality of application 102 may usethresholds to assist with determination and recognition process. Forexample, combinations of data exceeding specified levels may provide ahigh degree of confidence that the recognition process is accurate.

In embodiments, the machine learning functionality of application 102,teamed with the human user of a firearm, may be operated autonomously,for example, in response to a determined intent of the human user of thefirearm teamed with the machine learning functionality of application102. The machine learning functionality of application 102 may be usedto detect gestures of the human firearm user, for example, by capturingand analyzing data from sensors that detect conditions of the human, aswell as firearm sensors. Sensors that detect conditions of the human mayinclude multi-modal sensors and multi-modal wearable sensors. Gesturesmay include pointing gestures, threat identification gestures, targetacquisition gestures, signaling gestures and the like.

In embodiments, conditions recognized by the machine learningfunctionality of application 102 or sensed in order to facilitatetraining of the machine learning functionality of application 102 mayinclude conditions indicative of human states, such as stress and otherphysiological states. Conditions indicative of human states and capturedby sensors for analysis by the firearm usage monitoring system mayinclude heart rate conditions, for example, physical staterelationships, blood pressure conditions, body temperature, galvanicskin response, heat flux, moisture, chemistry (for example glucoselevels), muscle states and neurological states. Various biologicalconditions or biosensors may be indicative of threats, such as heartrate conditions, body temperature, moisture (such as indicatingexcessive perspiration), blood pressure, galvanic skin response, andothers. Firearm sensors may be multi-modal firearm sensors and mayinclude sensors that detect motion, orientation and discharge state ofthe firearm.

In embodiments, the FAMS implements machine learning algorithms to forma motion-analysis model. Training data may be collected and curated froma set of data recorded by the FUMS. The training set may be formed bycleaning, organizing, and labeling the data. The cleaning includes, forexample, removing duplicative data, removing data that does not includethe target action, and removing false-positive data. The organizingincludes associating connected data from different sources. For example,the data may be structured such that sensor and other recordedinformation related to a single firearm is grouped together. Therecorded information may be from the firearm and coupled devices orexternal sources where the firearm is identifiable (e.g., surveillancevideo). The labeling includes assigning meaningful tags to the groupeddata, such as “discharge,” “no discharge,” “intentional,”“unintentional,” “misfire,” “jam,” “overheat,” “maintenance,” and otherrelevant labels. Information that is temporally proximate to the desiredlabels is also included within the training set. In some examples, thetemporally proximate data includes data from 10 minutes, 5 minutes, 3minutes, 1 minute, or 30 seconds prior to occurrence of the labeledevent and data from 1 second, 5 seconds, 30 seconds, 1 minute, 3minutes, or 5 minutes after the labeled event. The training set is thenprovided to a machine learning algorithm to form an analysis model thatis configured to be used in real-time to predict events during usage ofthe firearm (e.g., discharge or jamming). It can be shown that withpre-determined time intervals after a discharge event based on theweapon and its ammunition can provide a 99.7% identification rate. Inembodiments, the training set is further used to form a training modelconfigured to clean, organize, and/or label data to form one or moreupdated training sets. Beneficially, such models and training may beextended to learning and analysis of non-discharge patterns, such asdetermining movement patterns indicative of user conditions includingabnormal gait, injuries, over encumbrance, cognitive impairment, andexhaustion.

Analyzing the data by application 102 (e.g., by firearm monitoringmodule 400, threat detection and analysis module 402, threat responsemodule 404, GUI generation and display module 406, signal processingmodule 408, or another software module of application 102) may produce aset of candidate intents of the human firearm user or of anotherindividual in proximity to the firearm user (such as where camerainformation, voice information, and the like is available). Thecandidate intents may, in embodiments, be combined with physical andoperation machine state information to select one or more action plans.The machine teamed with the human user of the firearm may then executeand adjust the selected action plan based on updated intents, machinestates, and environmental factors. Machine state factors may includephysical factors, operational factors, orientation factors,tactile/force factors, and the like.

Environmental factors may include weather factors, location datafactors, altitude factors, topography factors, video factors and thelike. Weather factors may include temperature, humidity, wind speed,wind direction and precipitation factors, among others. Location datafactors may include streaming data, as well as data acquired fromgeolocation services (e.g., using a global navigation satellite system,for example, GPS, GLONASS, BDS, Galileo, or the like) and beacons,connection points or the like, as well as through cellular. Topographyfactors may include data and observations, while video factors mayinclude both live and archived video feeds. The action plan may also beformed from a set of predetermined action steps, for example, actionsteps that each satisfy human teaming criteria selected to coordinatewith at least one of the candidate intents. Actions steps may also bearranged into action plans by sets of rules.

In embodiments, the machine learning functionality of application 102may be trained to recognize and distinguish between non-combatactivities and combat activities. For example, the machine learningfunctionality of application 102 may be trained to recognize celebratorysituations such as dancing scenarios and first bump scenarios separatefrom other human machine learning scenarios in much more threatening andcomplex environments. In other examples, the machine learningfunctionality of application 102 may be trained to distinguish betweencelebratory fire and threatening fire. By way of these examples, themachine learning functionality of application 102 may learn themovements of the users of system 100, for example, by translating anddetecting their motion and comparing the identified motions in contextwith a deployment location in comparison with trained examples,confidence in those examples, corrections to past activity, and the liketo assist, anticipate, protect, support, and facilitate the needs of theusers in the theater more quickly and more safely.

In embodiments, the machine learning functionality of application 102may manage a coordinated team of human users of firearms and at leastone machine. In this embodiment, the machine learning functionality ofapplication 102 may receive as inputs at least one sensory input about ahuman and at least one sensory input about a machine that is part of theteam coordinated with the human. The machine learning functionality ofapplication 102 may then automatically, using machine learning,determine the occurrence of an event, such as a pre-discharge event, adischarge event, a post-discharge event (including a post dischargeadverse event) or other events. Post discharge adverse events mayinclude injury to the human or occurrence of damage to the machine, suchas subsequent to the detection of a firearm discharge event by thesystem.

In embodiments, application 102 (e.g., using firearm monitoring module400 or another module) may track with a global navigation satellitesystem (e.g., GPS). In embodiments, application 102 includes networkreporting facility, such as through a Bluetooth® or other short- orlong-range discharge report to a centralized server.

The functions of the system sub-components shown in FIG. 4 can beimplemented in digital electronic circuitry, in computer software,firmware or hardware. The techniques can be implemented using one ormore computer program products. Programmable processors and computerscan be packaged or included in mobile devices. The processes may beperformed by one or more programmable processors and by one or more setof programmable logic circuitry. General and special purpose computingand storage devices can be interconnected through communicationnetworks.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in amachine-readable or computer-readable medium (alternatively referred toas computer-readable storage media, machine-readable media, ormachine-readable storage media). The computer-readable media may store acomputer program that is executable by at least one processing unit andincludes sets of instructions for performing various operations.Examples of computer programs or computer code include one or more of:source code; object code; machine code, such as is produced by acompiler; or files including higher-level code that are executed by acomputer, an electronic component, or a microprocessor using aninterpreter.

Referring to FIG. 5 , inputs 500, processing options 502, and outputs504 related to threat detection and analysis functionality of system 100are shown by example. Inputs 500, processing options 502, and outputs504 may, for example, refer to functionality of threat detection andanalysis module 402. Inputs 500 shown by example include sensors 506,knowledgebase 508, and machine learning models 510. Processing options502 shown by example include firearm discharge detection 512, firearmorientation change detection 514, asset motion detection 516, andnon-user motion detection 518. Outputs 504 shown by example include nothreat 520, detected threat 522, and threat severity 524. One or moreinputs 500 may be used to determine one or more outputs 504 using one ormore processing options 502.

Referring to FIGS. 6-10 , example GUIs of application 102 are shown. TheGUIs shown in FIGS. 6-10 may, for example, be GUIs generated anddisplayed using GUI generation and display module 406. In FIG. 6 ,top-down geographic view 600 representing a satellite-view visualizationof a deployment location is shown. Users 602A-D are shown at particularlocations within the deployment location, for example, based ongeolocation sensors included within or coupled to firearms or othermobile assets of users 602A-D. Cones of fire 604A-D are shown asprojecting outwardly from respective ones of users 602A-D in directionsin which firearms of those ones of users 602A-D are pointing. Inparticular, cones of fire 604A-D are shown in FIG. 6 as non-overlapping.As will be understood, depending on the orientations of firearms ofusers visually represented within top-down geographic view 600, cones offire may be non-overlapping, partially overlapping, or whollyoverlapping. The greater the overlap and the greater numbers of overlapindicate a higher likelihood of a present threat since the firearms arebeing aimed at a common location. Top-down geographic view 600 ispopulated with icons showing exact locations of firearms 104. Inembodiments, the icons can include all personnel and/or statusinformation for the firearms. In embodiments, the icons can include abutton or other user interface element used to zoom in on the locationof a firearm (e.g., to drill down on data associated with the firearm).

Also in FIG. 6 , three-dimensional firearm orientation view 606,two-dimensional recoil tracking view 608, and user body camera feed view610 are shown. Three-dimensional firearm orientation view 606 representsa visualization of a firearm of a user (e.g., one of users 602A-D) withreal-time updates based on the specific orientation of the firearm.Two-dimensional recoil tracking view 608 represents a visualizationshowing real-time changes over time of an orientation of a barrel orother portion of a firearm based on recoil resulting from firing thefirearm. In embodiments, the visualization showing real-time changesover time of an orientation of a barrel or other portion of a firearmcan also be based on other displaying motions such as transitioning todifferent targets, over or under adjustment for target transitions,pre-shot and post-shot movement or jitter. User body camera feed view610 represents a real-time video stream from a camera or other imagingdevice worn by a user or otherwise included within or coupled to anasset on a user. In embodiments, one or more of three-dimensionalfirearm orientation view 606, two-dimensional recoil tracking view 608,or user body camera feed view 610 may not be included in the GUI whichincludes top-down geographic view 600. In embodiments, the GUI whichincludes top-down geographic view 600 may include views other thanthree-dimensional firearm orientation view 606, two-dimensional recoiltracking view 608, and user body camera feed view 610.

In embodiments, top-down geographic view 600 or a GUI which includestop-down geographic view 600 can display notifications providing detailsabout one or more of users 602A-D, cones of fire 604A-D, or informationrelating to one or more of views 606, 608, or 610/For example, thenotifications can indicate information regarding movements of a firearmrelative to a user thereof, for example, as “weapon aimed,” “weaponholstered,” “weapon separated from the user,” and the like.

In embodiments, information about some or all of users 602A-D may bedisplayed in the GUI. The information may, for example, include orrelate to names, ranks, years of service, skill levels, weapons present,ammunition stocks present, numbers of shots fired since arrival at thelocation shown, health information, threat engagement information, orthe like, or a combination thereof. In some such embodiments, theinformation may be displayed in the GUI by default. In other suchembodiments, the information may be displayed in response to aninteraction by a remote user of the GUI. For example, information for agiven user may be displayed as a prompt in response to the remotedashboard user selecting that given user within the GUI.

In embodiments, information representative of sensor measurementsrecorded using some or all of the sensors within the deployment locationmay be displayed in the GUI. The information may, for example, includeor relate to sensor types, measurements, flags which indicate that themeasurements represent actionable information (e.g., a trigger sensormeasurement indicates that the user's finger is on the trigger and/orthat the trigger has been toggled, such that a threat engagement isunderway), or the like, or a combination thereof. In some suchembodiments, the information may be displayed in the GUI by default. Inother such embodiments, the information may be displayed in response toan interaction by a remote user of the GUI. For example, information fora given firearm within the deployment location may be displayed as aprompt in response to the remote user of the GUI selecting that givenfirearm or the user thereof within the GUI. In embodiments, some or allof the projections or orientation views can be shown inthree-dimensional renderings.

In FIG. 7 , top-down geographic view 600 of FIG. 6 is shown. Here, thecones for the firearms of the users visually represented in the top-downgeographic view are shown as coalescing. For example, whereas thepositions and orientations of cones of fire 604A-B and correspondingusers 602A-B in FIG. 6 were based on first sensor information collectedat a first time, the positions and orientations of cones of fire 604A-Band corresponding users 602A-B in FIG. 7 are based on second sensorinformation collected at a second time after the first time. The GUIincluding top-down geographic view 600 as shown in FIG. 7 has thus beenupdated as compared to how that GUI appears in FIG. 6 . The coalescence(e.g., partial or whole overlap) of multiple one of cones of fire 604A-Bis used to detect a threat 700. For example, the coalescence of themultiple cones of fire 604A-B may indicate that multiple users 602A-Bassociated with those cones of fire 604A-B are actively drawing theirfirearms on, or otherwise towards, threat 700. In embodiments, acoalescence of cones of fire can be visually represented in top-downgeographic view 600 by changing an appearance of the coalescing cones.For example, coalesced cones can be changed to visually appear in adifferent color (e.g., from white to red), with shading, with differentborder line thickness, or in another emphasizing manner. In embodiments,a severity of a detected threat may be visually represented in top-downgeographic view 600. For example, a higher severity may be visuallyrepresented at the icon of threat 700, for example, by changing a color,shading, border thickness, or other aspect of the icon of threat 700.

In FIG. 8 , the top-down geographic view 600 of FIG. 6 is again shown,but with visual prompts 800 and 802 representing information relating tousers 602A and 602B, respectively, and visual prompt 804 representinginformation relating to threat 700. Prompt 800 visually representswithin top-down geographic view 600 that user 602A is named S. Smith, isskill level ten, and has low ammunition supply. Prompt 802 visuallyrepresents within top-down geographic view 600 that user 602B is namedM. Matthews, is skill level five, and has low ammunition supply. Prompt804 visually represents within top-down geographic view 600 that threat700 is an actively engaged threat and includes four or more hostiles. Inembodiments, one or more of visual prompts 800, 802, or 804 isautomatically shown in the GUI which includes top-down geographic view600. In embodiments, one or more of visual prompts 800, 802, or 804 isshown in the GUI which includes top-down geographic view 600 in responseto selection by a user of application 102 (e.g., a remote support user)of one or more of user 602A, user 602B, or hostile 700 within top-downgeographic view 600. In embodiments, one or more of visual prompts 800,802, or 804 may be visually represented within the GUI which includestop-down geographic view 600, but outside of top-down geographic view600. For example, a separate view of that GUI may present text-based orother information about users 602A-B and/or about threat 700.

In FIG. 9 , a top-down geographic view 900 different from top-downgeographic view 600 is shown. Top-down geographic view 900 represents anoverhead visualization of multiple users 902 engaged in live fire 904.In embodiments, top-down geographic view 900 displays informationrelating to one or more of sectors of fire 906, threat locations 908,weapon statuses 910, fellow and partner forces 912, or ammunitionstatuses 914. In embodiments, sectors of fire 906 can show the areabeing attacked. By way of this example, a cone of other suitable shapescan be depicted adjacent to the weapon to show the area to which livefire is directed. It will be appreciated that movements of the weaponand movements of users 902 motivate showing sectors of fire 906 incone-like shapes rather than lines. In doing so, these sectors of fire906 can overlay on each other when there is multiple live fire fromfellow and partner forces 912 and their intersections can identify orfacilitate in the identification of threat locations 908. Inembodiments, weapon statuses 910 and ammunition statuses 914 canindicate whether ammunition is running low, time until exhaustion ofammunition, jammed weapon, and the like. In embodiments, top-downgeographic view 900 may visually represent one or more of sectors offire 906, threat locations 908, weapon statuses 910, fellow and partnerforces 912, or ammunition statuses 914 using icons. In some suchembodiments, top-down geographic view 900 or a GUI which includestop-down geographic view 900 may include a legend of those icons. Inembodiments, ammunition statuses 914 may be visually represented indifferent ways based on the status. For example, an ammunition statusindicative of the user having a sufficient ammunition inventory may beshown by green ammunition status 914A. In another example, an ammunitionstatus indicative of the user having an ammunition inventory which isrunning low (e.g., lower than a threshold, which may be configurable ordefined based on the firearm type or otherwise) may be shown by yellowammunition status 914B. In yet another example, an ammunition statusindicative of the user having an ammunition inventory which is depletedor nearly depleted may be shown by red ammunition status 914C.

In FIG. 10 , first dashboard view 1000 representing a visualization ofan example of a first page of a dashboard of application 102 is shown.Application front-end includes pages corresponding to tabs 1002, 1004,1006, and 1008. Tab 1002 corresponds to a main page (e.g., the firstpage shown in FIG. 10 ), Tab 1004 corresponds to a maps page (e.g., fordisplaying a GUI with top-down geographic view 600 shown in any of FIGS.6-9 ). Tab 1006 corresponds to a threats page (e.g., described belowwith respect to FIG. 11 ). Tab 1008 corresponds to a knowledgebase(e.g., representing data stored in database 124). First dashboard view1000 includes user interface elements 1010, 1012, and 1014 for reportinginformation about system 100. Element 1010 reports a list of userscurrently deployed within a deployment location (e.g., users 60A-B).Element 1012 reports a list of communication systems in use. Element1014 reports a list of alerts for the attention of a user of thedashboard.

In embodiments, as in the example shown in FIG. 10 , the informationreported using elements 1010, 1012, and 1014 is organized by groups ofusers. However, in embodiments, such information may be organized byindividual user or in another manner. In embodiments, element 1012 mayinclude status information for connections between application 102 andconnection points deployed within deployment locations.

In FIG. 11 , second dashboard view 1100 representing a visualization ofan example of a second page of the dashboard of application 102 isshown. In the example shown, the second page as shown in FIG. 11 is thethreats page corresponding to tab 1006. Second dashboard view 1100includes user interface elements 1102, 1104, 1106, 1108, and 1110.Elements 1102 and 1104 are used for reporting information about system100. Elements 1106, 1108, and 1110 are interactive elements which can betoggled (e.g., clicked) by a user of the dashboard. Element 1102 reportsa list of detected threats, including information about users affectedby the threats, predictions of what the threats are, and conditionsfaced by the users in addressing the threats. Element 1104 reports alist of automated responses taken based on the detected threats,including methods of performance and estimated times for performance.Element 1106 is a button allowing a user of the dashboard to send analert relating to the threats reported in element 1102 and/or theresponses reported in element 1104, for example, to users deployedwithin the deployment location related to the detected threats or toothers (e.g., remote managers or other personnel). Element 1108 is abutton allowing a user of the dashboard to modify an automated responsereported in element 1104, for example, if he or she believes differentsupport would be useful or based on communications from the usersengaged in addressing the threat (e.g., after the detection of thethreat). Element 1110 is a button allowing a user of the dashboard tocancel an automated response reported in element 1104, for example, ifhe or she believes support is no longer necessary or based oncommunications from the users engaged in addressing the threat (e.g.,after the detection of the threat and/or after a portion of theautomated response is performed).

In embodiments, threats and responses reported within second dashboardview 1100 may correspond to any of a number of user groups registeredwith system 100. In embodiments, threats and responses reported withinsecond dashboard view 1100 may correspond to individual groups. Inembodiments, threats and responses reported within second dashboard view1100 may correspond to individual users. In embodiments, the responsesreported within element 1104 are not automated. In such embodiments, thesecond dashboard view 1100 includes a user interface element for theuser to interact with to verify or otherwise approve a proposedresponse.

In embodiments, the number and/or types of elements included within aGUI or within a view of a GUI (e.g., a GUI as shown in any of FIGS. 6-11) can be controlled based on the type of GUI or the type of view withinthe GUI. For example, when top-down geographic view 600 becomes toodense with overlapping icons, the GUI which includes top-down geographicview 600 may automatically update to visually represent a new iconsymbolizing multiple units within the area shown. For example, referringto FIG. 6 , users 602A-B may be combined into a user group and visuallyrepresented using a single icon rather than the two separate iconsshown.

In embodiments, the dashboard of application 102 may include GUIs and/orviews other than what is shown in FIGS. 10 and 11 . For example, thedashboard of application 102 may include communication and mappingfeatures, such as to track the location of all weapons in real-time, tohighlight relevant events (such as weapons being gripped, weapons beingraised, or weapons that have been discharged). In some such embodiments,the dashboard of application 102 may include the GUIs shown in any ofFIGS. 6-9 (e.g., accessible via tab 1004 or another tab or userinterface element). In another example, the dashboard may provide accessinformation from other systems, such as making available camera views,such as ones that are triggered by activation of body cameras or on-sitecameras from within the deployment location or from the dashboard. Inembodiments, the dashboard includes a GUI and/or view for separatingusers into groups/echelons with designated permissions. For example, thedashboard may include different GUIs and/or views for each of one ormore of ground units, officers, military personnel, aninvestigator/compliance officer, and the like.

Referring to FIG. 12 , a mesh network usable with system 100 is shown.The mesh network is made up of a number of network devices, includingconnection points 1200 and 1202, either of which may, for example, beconnection point 116. The mesh network is used by a number of users(e.g., some or all of users 1204, 1206, 1208, 1210, and 1212) tocommunicate information (e.g., sensor measurements) recorded for thoseusers to the server device 112 for processing using application 102. Inembodiments, some of the users, such as users 1204, 1206, and 1208, havemobile computing devices 1214, 1216, and 1218, respectively, whichoperate to extend the mesh network when in range of connection point1200 or connection point 1202.

Depending on their proximity to connection point 1200 or to connectionpoint 1202, devices associated with users 1204, 1206, 1208, 1210, and1212 may or may not be able to connect to connection point 1200 or toconnection point 1202. In the example shown, users 1208 and 1210 arestanding in the range of connection point 1200 and therefore devices ofusers 1208 and 1210 may be able to connect to connection point 1200.Similarly, user 1210 is shown as standing in the range of connectionpoint 1202 and the devices of user 1210 may be able to connect toconnection point 1202. In embodiments, the range of the mesh networkcreated by connection points 1200 and 1202 may be extended by mobilecomputing devices 1214, 1216, and 1218. For example, in such anembodiment, users 1204 and 1206 would also be within the range ofconnection point 1200 and therefore devices of users 1204 and 1206 maybe able to connect to connection point 1200. However, because user 1212is not within the range of either connection point 1200 or connectionpoint 1202, and because user 1212 does not have a mobile computingdevice to extend the mesh network created thereby, devices associatedwith user 1212 are unable to connect to either connection point 1200 orconnection 1202 while user 1212 remains at the location shown.

In embodiments, the mesh network may be a self-organizing and fluid meshnetwork that organizes and reorganizes itself based on specified data,including data filtered or weighted based on specified criteria, and/orthe dynamic detection of other devices, for example with a geographicperimeter. Other devices may include deployable mesh network hubs (alsoknown as “pucks”), beacons, wireless connection points, such as Wi-Ficonnection points, lighting systems, cameras, and the like, each ofwhich may be connection point 1200 or connection point 1202 or mobilecomputing device 1214, 1216, or 1218. The mesh network may also includeasset management systems, crowdsourced communications, frequencyscanning networking systems, cellular mesh networking systems, and/orother systems.

In embodiments, devices on the mesh network may adjust locationinformation based on the relative movement of each other within the meshnetwork. In embodiments, the relative movement of devices may bereported by other devices within the mesh network over the mesh network,such as to the self-disposing devices. The relative movement of otherdevices may also be derived from IMUs disposed with the other deviceswithin the mesh network.

Relative movement information may include speed, velocity, accelerationor position information, and/or event identification information. Suchinformation may include threat identification information, shot accuracyinformation and the like. Event identification information may includeweapon information, information indicating a person is in anunauthorized area, soldier maneuver information (e.g., speed, direction,activity, or the like), in-position information (such as for anindividual or a device), rate-of-fire information, alternating fireinformation, maintenance required information, stoppage eventinformation, ammunition expenditure information, fight or struggleinformation and the like. In embodiments, authentication information maybe received from RF identification (RFID) implants, for example,implanted in the person.

In embodiments, the relative movement, such as among devices in the meshnetwork like firearms and other equipment may be provided relative to atleast one geographic location, such as through the use of data from theIMUs or from one or more other data sources. In embodiments, locationmay relate to relative locations of one or more other firearms or otherdevices connected to the mesh network, such as the distance, direction,and/or movement of one or more other firearms or other devices relativeto a given one. In such embodiments, geographic location and movementinformation, whether relating to a location or to another firearm orother device may be communicated to a given firearm or other systems ofan individual handling a firearm over the mesh network. In embodiments,the geographic location may be an underground geographic location, whereother geographic location detecting signals, such as GPS are notavailable. In embodiments, a combination of geographic location andrelative location may be understood by the system, such as where atleast one member of a mesh network has a detectable location (such as byGPS signal) and other members have locations that are determinedrelative to the known member, such as by detecting motion through theIMU or other non-GPS systems. It may be appreciated from theseembodiments that using data from the IMU on the mesh network may allowthe firearm usage monitoring system to provide discharge locationinformation in geographic locations that may not otherwise be covered bygeographic location detecting signals.

In embodiments, the mesh network connection may be a wireless meshnetwork connection and may be configured based on radio communicationfrequencies. In some situations, radio communication frequencies may besubject to interference or jamming, either intentionally or otherwise,making communication difficult or impossible when attempting toestablish a connection over the compromised frequency. Interference orjamming may include radio frequency interference or jamming, opticaljamming, noise, and the like. Because of the risk of jamming, andbecause communication reliability may be critical for the user of system100, the firearm usage monitoring system may detect such jamming of oneor more frequencies and automatically adjust the frequency of the meshnetwork to avoid using the compromised frequency, such as by selecting afrequency not currently subject to interference or jamming. System 100may then establish a wireless mesh network connection with anotherdevice using the selected frequency. Jamming or interference detectionmay include detecting attempted signal interception and scramblingtransmitted information to avoid the detected signal interception.

In embodiments, system 100 may determine discharge information relatedto the firing of a firearm of one of users 1204, 1206, 1208, 1210, or1212 connected to the mesh network. The discharge information mayinclude discharge location, direction of the discharge, a motion path ofthe firearm preceding discharge and/or orientation of the firearm atdischarge. Orientation information may be provided by the IMU of thefirearm and may include enemy area location and size information, unsafeact information, line of fire information, shift fire information,sectors of fire information, interlocking fire information, 360perimeter security information and the like.

The discharge information may be determined from motion and locationinformation, such as provided by devices connected to the mesh network.For example, the discharge location may be determined from geographiclocation data of one or more firearms connected to the mesh network andmay use relative movement data provided by the other devices connectedto the mesh network, for example by analyzing relative movement datathat is based on resident IMU data from other firearms connected to themesh network.

In embodiments, system 100 may perform over-the-air updates for hardwareand/or software within the range of connection points 1200 and 1202using connection points 1200 and 1202. In embodiments, devices withinrange of connection points 1200 and 1202 may be charged by wirelesscharging using connection points 1200 and 1202 as the power sources. Inembodiments, connection points 1200 and 1202 may record data (such asIMU data) from devices within range thereof when those devices are inactive or inactive modes (such as to flash memory) and may enable othermodes, such as a sleep/hibernation mode.

In embodiments, system 100 may function in active modes, sleep modesand/or hibernation modes. In the active mode, a device (e.g., componentsof a firearm, a computing device such as devices 1214, 1216, 1218, orthe like, or a device of another asset) may be in full power mode, suchas using power for collecting readings from the IMU and GPS andtransmitting them via a local protocol like BLE to an edge device. Inembodiments, data can be sent in this format at relatively high datarates, such as at 30 messages/second, 50 messages/second, 100messages/second, or the like. A sample string may includeAB-FC-22-CC-B3-00-00-00-00-00-00-00-00-00-00-00-00-5E-89-5A-C0-71-3E-E6-C0-FA-18-9C-00-00-20-75-3F-00-80-52-3E-00-00-19-3E-00-00-64-40-67-66-00-C1-34-33-6B-00-01-BA.The guide may be as follows: AB (header), FC-22-CC-B3-00 (millisecondtimestamp), 00-00-00-00 (latitude), 00-00-00-00 (longitude), 00-00(altitude in meters), 00 (horizontal accuracy in meters), 5E-89-5A-C0(gyro x), 71-3E-E6-C0 (gyro y), FA-18-9C-C0 (gyro z), 00-20-75-3F (accelx), 00-80-52-3E (accel y), 00-00-19-3E (accel z), 00-00-64-40 (mag x),67-66-00-CI (mag y), 34-33-6B-C0 (mag z), 01 (unit status), BA (footer).A millisecond timestamp may be used, such as in a modified UNIXtimestamp, e.g., for milliseconds after 01-01-16. In embodiments, if BLEis unavailable or a message is not sent, this may be stored in the flashmemory to be sent when the device enters sleep mode. The active modemay, for example, be triggered when force is applied to a force sensor.Depending on the configuration, a device may remain in the active modefor a specified time, such as two minutes after the force is no longerapplied, for five minutes, for ten minutes, or the like. This timer maybe reset when force is reapplied.

In embodiments, devices connected within range of connection point 1200and/or connection point 1202 may also power down into a “sleep” mode,such as when there is no longer force applied to the device and thetimer has gone down (indicating expiration of active mode). In such asleep mode, one message may be sent at a defined period, such as onceper second, such as containing the timestamp, location data, and currentorientation data. A GPS module or like component of the device may enteran ATP (adaptive trickle power) state where it cycles between full powerand ATP to minimize power consumption while maintaining a fix on itslocation. In embodiments, a location fix may be maintained consistently,regardless of power mode. In embodiments, the IMU may be polled at a lowrate, such as to monitor movement. If no movement is sensed for a giventime, such as five minutes, then the unit may go into another even lowerpower mode, referred to herein as a hibernation mode. In some suchembodiments, connection points 1200 and 1202 may monitor and selectivelycontrol changes in modes of devices, for example, based on timestampsindicating connections between the devices and ones of connection points1200 and 1202, based on signals received by connection points 1200and/or 1202 from ones of the devices, and/or based on other criteria.

In such a hibernation mode, a device may continue to send messages(e.g., one per second), such as containing the timestamp, location data,and current orientation data. The GPS module may enter hibernation whereit consumes, for example, under 1 mA of power. The IMU may still bepolled at a low rate. If movement exceeds a certain threshold, the unitmay go into sleep mode and the GPS module may wake up to maintain alocation fix. This mode may consume, for example, under 7 mAh.

In embodiments, the firearm 104 further includes sensors 118, acommunication interface, a buffer, and a controller.

The communication interface is configured to pass data between devicesor components coupled thereto, such as the firearm 104 and a connecteddevice (e.g., wearable devices 106, stationary device 108, connectionpoint 116). The communication interface may include a suitable number ofconductors, connectors, transmitters, and/or receivers to achievedesired data throughput and device connectivity. The communicationinterface may communicate with devices and components through wiredand/or wireless telecommunication protocols, such as ethernet,transmission control protocol (TCP), Internet protocol (IP), power linecommunication, Wi-Fi, Bluetooth®, infrared, radio frequency (RF),general packet radio services (GPRS), global system for mobilecommunications (GSM), frequency-division multiple access (FDMA),code-division multiple access (CDMA), evolution-data optimized (EVDO),Z-Wave, ZigBee, 3G, 4G, 5G, another protocol, or a combination thereof.

The buffer is configured to temporarily store data received from, forexample, sensors 1302 prior to transmission via a signal medium orwriting to a storage medium. The buffer may be a suitable physical orvirtual medium. The buffer may operate in a suitable manner, such as byimplementing a first-in, first-out queue where the oldest data is thefirst data read out of the buffer for transmission or storage. Inexamples, the oldest data or subsets of data in the buffer may beoverwritten by the newest data being read into the buffer. Beneficially,the buffer may provide for reduced power consumption based onimplementation by optimizing compute time and transmission payload size.The buffer may be a static allocation of a suitable size or may bedynamically allocated. Beneficially, dynamic allocation allows thefirearm usage monitoring system 2800 to adjust the buffer allocationbased on or in response to triggering events or predeterminedconditions, which thereby improves operation of the firearm usagemonitoring system 2800 by optimizing resource allocation. For example,if long-term storage resources onboard the firearm 102 are approachingcapacity, the size of temporary storage in the buffer may be reduced andthe freed space may be used for long-term storage until connection isreestablished or the long-term-storage data is further processed and/orreduced in size.

The controller is configured to run application software that controlsoperation of connected components. The terms “controller,” “controlmodule,” “control,” “control unit,” “processor” and similar terms meanApplication Specific Integrated Circuit(s) (ASIC), electroniccircuit(s), central processing unit(s) (preferably microprocessor(s))and associated memory and storage (read only, programmable read only,random access, hard drive, etc.) executing one or more software orfirmware programs or routines, combinational logic circuit(s),sequential logic circuit(s), input/output circuit(s) and devices,appropriate signal conditioning and buffer circuitry, other components,combinations thereof, and the like to provide the describedfunctionality. “Software,” “firmware,” “programs,” “instructions,”“routines,” “code,” “algorithms” and similar terms mean controllerexecutable instruction sets including calibrations and look-up tables.In some aspects, the controller includes a central processing unit(CPU).

To appropriately control operation of coupled components, the controllermay include a processor (e.g., a microprocessor) and at least onememory, at least some of which is tangible and non-transitory. Thememory can store controller-executable instruction sets, and theprocessor can execute the controller executable instruction sets storedin the memory. The memory may be recordable medium that participates inproviding computer-readable data or process instructions.

The recordable medium may take many forms, including but not limited tonon-volatile media and volatile media. Non-volatile media for thecontroller may include, for example, optical or magnetic disks and otherpersistent memory. Volatile media may include, for example, dynamicrandom-access memory (DRAM), which may constitute a main memory. Thememory of the controller may also include a solid-state medium, a floppydisk, a flexible disk, hard disk, magnetic tape, another magneticmedium, a CD-ROM, DVD, another optical medium, combinations thereof, andthe like.

The controller-executable instruction sets may be transmitted by one ormore transmission media, including coaxial cables, copper wire ortraces, fiber optics, combinations thereof, and the like. For example,the transmission media may include a system bus that couples two or morecomponents of the firearm monitoring system 3500, such as thecontroller, the communication interface, and the sensors.

The controller can be configured or equipped with other requiredcomputer hardware, such as a high-speed clock, requisiteAnalog-to-Digital (ND) and/or Digital-to-Analog (D/A) circuitry,input/output circuitry and devices (I/O), as well as appropriate signalconditioning and/or buffer circuitry. Any algorithms required by thecontroller or accessible thereby may be stored in the memory andautomatically executed to provide the required functionality for therelevant components, such as the buffer, the communication interface,and the sensors.

In embodiments, the application software is configured to operate thefirearm 104 and/or components thereof in a plurality of states, as wellas being configured to detect conditions related to such operation. Theplurality of states may include, for example, one or more firearm-sleepstates, one or more cloud-constrained states, one or more situationalstates, combinations thereof, and the like.

The one or more sleep states are configured to reduce resource impact ofthe firearm 104. The resource impact may include, for example, powerusage, data storage, and/or data transfer. The sleep states may include,for example, a storage state, a hibernation state, a standby state, ahybrid-standby state, combinations thereof, and the like.

Beneficially, the storage state may provide reduced power consumptionand a reduced resource footprint while the firearm 104 is being stored.For example, operation of components of the firearm usage monitoringsystem 2800 disposed on or attachable to the firearm 104 may beeliminated or significantly reduced while those components are instorage.

In aspects, the controller is configured to power down components of thefirearm 104 such as sensors 1302 and storage media, wake, in response tofulfilment of a predetermined condition, a location sensor of thefirearm 104, and determine a location of the firearm 104. In response todetermining the firearm 104 is still in the storage location, thecontroller is configured to return to the powered-down state. Inresponse to determining the firearm 104 is outside the storage location,the controller is configured to operate the firearm 104 in adata-acquiring state, such as a situational state. Optionally, thefirearm usage monitoring system 2800, in response to determining thefirearm 104 is still in the storage location, may further acquire data,record acquired data to a long-term storage medium onboard, transmitacquired data to a remote device prior to returning to the powered-downstate, receive data from a remote device, combinations thereof, and thelike.

The predetermined condition may be, for example, a predetermined periodof time elapsing, detection of a mechanical event such as a switchactuation, combinations thereof, and the like. In aspects, the period oftime may be a day, a week, a month. In aspects, the interval ofsubsequent periods of time may be increased. For example, the period oftime may be a day until the firearm 104 has remained in the storagestate for a week in the sleep state, then the period of time may beincreased to a wake-up every week until the firearm 104 has remained inthe storage state for a month, and then the period of time may beincreased to a wake-up every month.

In aspects, the storage state optimizes power consumption duringlong-term non-use of the firearm 104 while also providing for inventorytracking and/or management of the firearm 104. Optionally, the storagestate may also provide security features for the firearms 104, such asinhibiting usage of the firearm 104 and preventing firearm 104trafficking.

Beneficially, the hybrid standby state may provide for switching fromthe standby state to the hibernate state without additional resourceoverhead.

In aspects, the controller is configured to power down components of thefirearm 104, and, while entering the sleep mode, write relevant data tonon-volatile memory. The controller may be further configured to monitora condition of the firearm 104, wait a predetermined period of time, andswitch, in response to the predetermined condition not being fulfilledat an end of the predetermined period of time, to a hibernation modewithout writing previously acquired data to non-volatile memory. What ismore, if a timestamp for entering hibernation is desired, the firearmusage monitoring system 2800 does not need to write such a time uponentering the hibernation state. Instead, the firearm usage monitoringsystem 2800 may write the timestamp while or after exiting thehibernation mode because the predetermined period of time is known.Beneficially, this preserves data fidelity while further reducingresources required.

The predetermined condition may be, for example, a movement sensordetecting movement above a predetermined threshold, actuation of acomponent of the firearm 104, geospatial movement of the firearm,combinations thereof, and the like.

Notably, the hybrid standby mode reduces resource requirements becausethe firearm usage monitoring system 2800 does not need to expendresources to write previously acquired data when entering hibernationmode from sleep mode. For example, the firearm 104 may proceed to shutdown relevant components without establishing further networkcommunication or expending power to prepare components of the firearm104 for a data write.

The one or more cloud-constrained states are configured to maintain datafidelity and/or situational awareness while network connections orresources are limited or unavailable. The network conditions orresources may include, for example, communication bandwidth orinterference, communication denial, resource constraint imposed by aquality-of-service goal, combinations thereof, and the like. Thecloud-constrained states may include, for example, a network-constrainedstate, a network-blocked state, a cloud-compute-constrained state,combinations thereof, and the like.

Beneficially, the network-constrained state may optimize data transferfrom the firearm 104 through the network to minimize overall bandwidthrequired when connectivity issues are caused by terrain, physicaldistance from a connection point, interposing structures, combinationsthereof, and the like.

In aspects, the controller is configured to detect network-basedcommunication constraint and switch, in response to detecting thenetwork-based communication constraint, the firearm 104 to anetwork-constrained state including one or more of storing acquired datato non-volatile memory on the firearm, processing acquired data toreduce overall payload for delivery to the remote server, and adjustingcommunication intervals to optimize payload size for communication(e.g., increased packet size or minimized padding). The network-basedcommunication constraint may be detected, for example, through signalloss, packet losses, latency, combinations thereof, and the like.

Increasing memory allocation for storing data acquired by the sensorsmay include, for example, a dynamic allocation or static allocation. Theallocated block sizes may be determined based on sampling intervals ofthe sensors, predicted usage of the firearm 104, detected bandwidth ofthe network connection, combinations thereof, and the like.Beneficially, this preserves data fidelity while maintaining high-speed,low power access (e.g., values do not need to be read from storage intomemory) for rapid transmission when bandwidth is available again. Thisfurther increases the longevity of flash memory on the device.

Storing the acquired data to non-volatile memory may include, forexample, writing acquired data to flash memory. Beneficially, storingthe acquired data to non-volatile memory maintains data fidelity duringnetwork-constrained operation.

Reducing the data to be communicated may include, for example, applyingan algorithm to provide processed situational data in a smaller overallsize. For example, the algorithm may filter acquired data throughsuitable methods such as selecting a desired timestep that is greaterthan the acquisition timestep and preparing for transmission only datathat was acquired at the desired timestep, data of the greatest and/orleast magnitude during the desired time interval, an average value (suchas the mean, median, or mode) for the desired time interval,combinations thereof and the like. Beneficially, reducing the data to becommunicated provides for continued situational awareness whileoptimizing traffic through the network from both the firearm 104 andsimilarly connected firearms 104. Further, connection overhead may bereduced, for example, by reducing both request and response packets and,particularly the number of packets that must be resent.

Adjusting communication intervals may include, for example, increasingthe intervals for communication to an upstream connected device. Thetime interval may be adjusted based on data collection rate of thedevice, encryption protocol, communication protocol, frame size at thebottlenecked communication, combinations thereof, and the like.Beneficially, one or more of these values may be used to determine thecommunication interval which optimizes communication via, for example,increasing payload size and minimizing padding and message overhead(e.g., headers and footers needed to communicate the same amount ofdata) such that fewer packets are being communicated from the firearm104.

Optionally network-constrained state may further include sending amessage to the remote server to communicate the network-constrainedstate. Beneficially, the message promotes situational awareness of usersand commanders because both groups may be made aware of thenetwork-constrained state. When both groups are aware of thenetwork-constrained state, alternative methods of communication (e.g.,via an alternate frequency and/or communication media). Moreover,commanders may coordinate with additional units to support orcommunicate with the network-constrained user or users.

Additionally, or alternatively, the network-constrained state mayfurther include acquiring high-priority situational data andcommunicating the high-priority situational data to the remote server insubstantially real time. The high-priority situational data iscommunicated to the remote server while substantially all data acquiredprior to and after acquisition of the high-priority situational data isacquired, stored, and/or processed in accordance with standard operationof the firearm 104 in the network-constrained state.

The high-priority information may include, for example, a discharge ofthe firearm, actuation of a user-input mechanism (e.g., button, switch,or toggle), detecting a raising and/or aiming of the firearm 104,combinations thereof, and the like.

Beneficially, the network-blocked state may provide situationalawareness for commanders and users, as well as maintaining recordingand/or reporting ability of firearms 104 within the network-blockedlocation.

In aspects, the controller is configured to detect a network-blockedcondition and switch, in response to detecting the network-blockedcondition, the firearm 104 to a network-blocked state including one ormore of storing acquired data to non-volatile memory on the firearm,processing acquired data to reduce the storage footprint, establishingnon-blocked modes of communication to the remote server, and trackinggeospatial location and/or orientation using non-wireless mechanisms.

Network-blocked locations are locations where network communication issubstantially interfered with or denied.

Network communications interference may occur via unintentionalinterruption or disruption of signals for one or more forms of wirelesstransmissions (e.g., wireless communication or GPS signals). Forexample, structures, natural formations and phenomena, or presence ofother communicating devices may incidentally interrupt wirelesscommunication to or from devices at a particular location.

Denial of network communications may occur via deliberate interruptionor disruption of signals for one or more forms of wirelesstransmissions. For example, an actor opposing the user, such as a state,criminal actor, or hostile force, may deny network communication throughactive measures or passive measures.

Active measures include, for example, signal jamming or signalcapturing. Signal jamming may occur, for example, by one or more devicesemitting signals at one or more desired wavelengths to decrease thesignal-to-noise ratio at or near those wavelengths within the area ofeffect. Signal capturing may occur, for example, by one or more devicesconfigured to intercept signals emitted from a device (e.g., astingray). The intercepted signals may be re-emitted by the device orsunk.

Passive measures include, for example, selection of materials orlocations that impede wireless signals. Passive-denial materials mayinclude, for example, a metallic or metallized material that inhibitspropagation of signals therethrough (e.g., a Faraday cage).Passive-denial locations may include, for example, undergroundfacilities.

Detecting network-blocking may include, for example, the firearm 104,stationary devices, wearable devices, the remote server, combinationsthereof, and the like. In aspects, these devices may include aradio-communication system configured to receive and process wirelesssignals to determine network communications interference or denial. Forexample, a radio-communication system may be configured to receive andprocess signals to determine the presence of a jammer using suitablealgorithms. In aspects, the radio-communication system includes asoftware-defined radio operatively coupled to at least one antenna and acontroller. The antennas are configured to receive predeterminedfrequencies of wireless signals.

Additionally, or alternatively, the radio-communication system may beconfigured to receive and process signals to determine denial of networkcommunications using suitable algorithms. For example, the firearm usagemonitoring system 2800 may monitor communications signals between thefirearm 104 and remote server, calculate a signal power level profilewith respect to movement of the firearm 104, and determine, in responseto the power level profile either fulfilling a predetermined model ornot fulfilling predetermined models, that the firearm 104 is in anetwork-blocked location. In some aspects, the predetermined models maybe suitable attenuation models such as a log-distance path loss model orthe Hata model.

Establishing non-blocked modes of communication to the remote server mayinclude, for example, sweeping wireless frequencies to establishcommunication via a non-blocked wavelength, establishing optical orwired communication with proximate devices to backhaul data, forming anad-hoc network between firearms 104 and/or connected devices,combinations thereof, and the like.

In embodiments, the wireless-communications-blocked device or devices,such as firearms 104 or devices, may traverse communications frequenciesvia, for example, hopping across or sweeping through predeterminedcommunications frequencies to establish communications over anon-blocked frequency. For example, the wireless-communications-blockeddevice may ascend frequencies, descend frequencies, alternatingly ascendand descend frequencies, proceed through frequencies in a known order,combinations thereof, and the like, such that the probability ofestablishing communications via a non-blocked channel is above apredetermined threshold. In embodiments, the probability of establishinga communication connection is above 75%, more preferably above 90%, yetmore preferably above 99%.

The predetermined ranges may be, for example, one or more suitablefrequency domains for communicating the desired information. Inembodiments, the frequency domains include wireless-standard domainssuch as Wi-Fi bands, Bluetooth bands, Cellular bands, etc.

In embodiments, the wireless-communications-blocked device traversesfrequencies, analyzes whether the communications signal is indicative ofa jamming signal, and removes, in response to detection of the jammingsignal, the frequency from the frequencies being traversed.

In embodiments, the wireless-communications-blocked device is configuredto transmit data on low-bandwidth frequencies and/or short-rangefrequencies. Beneficially, the data transmitted on the low-bandwidthfrequencies may communicate a shared frequency within the desiredwireless-communications band having sufficient bandwidth to communicatethe situational data.

In embodiments, the wireless-communications-blocked device beginstraversing frequencies in response to a predetermined user input. Forexample, two users with non-blocked communications (e.g., visual,audial, or radio communications) may synchronize actuation andinitiation of the algorithms for frequency traversal.

In embodiments, the wireless-communications-blocked device, isconfigured to establish an ad-hoc network such that device-to-devicesignal communication is above a predetermined signal-to-noise ratio. Thead-hoc network may be, without limitation, a mesh network or serialnetwork. For example, a mesh network of wireless-communications-blockeddevices may be formed and movement of users of those devices may beconfined such that the distance between adjacent nodes is maintainedbelow the desired threshold to maintain a signal-to-noise ratio abovethe desired amount.

Additionally, or alternatively, a serial network ofwireless-communications-blocked devices may be formed (e.g., a “daisychain”) and movement of users of those devices may be directed such thatthe distance between adjacent nodes maintains a signal-to-noise ratioabove the desired amount while extending range of the users into thenetwork-blocked location. For example, a seven-member unit may establisha daisy-chain network where each user is connected to the adjacent user(e.g., user 1 is connected to user 2, user 2 is connected to users 1 and3, user 3 is connected to user 2 and 4, user 7 is connected to user 6and the external network, etc.). When the unit begins clearing thenetwork-blocked location, the unit may enter until the signal-to-noiseratio of the connection between user 7 and connection to the externalnetwork (e.g., connection point) approaches or reaches a predeterminedfloor. At that point, user 7 may remain generally stationary while theremaining unit members continue clearing the network-blocked locationuntil the signal-to-noise ratio of the connection between user 6 anduser 7 approaches or reaches a second predetermined floor. This processmay continue until the signal-to-noise ratio of the connection betweenuser 1 and user 2 approaches or reaches a final predetermined ratio. Inembodiments, the predetermined floors for signal-to-noise ratios mayprogressively decrease such that the available bandwidth is able toaccommodate backhaul for data from all downstream devices. Beneficially,the serial network of wireless-communications-blocked devices maximizespenetration of the unit into the network blocked location whilemaintaining interconnectivity of devices within the unit, as well asoptionally maintaining connection of the devices to the externalnetwork. Additionally, processing overhead is reduced and battery lifeincreased by establishing the serial network described above because thenodes within the network do not need to allocate compute time or powertoward detecting and establishing newer, stronger connections.

Tracking geospatial location and/or orientation using non-wirelessmechanisms may include, for example, use of an IMU, electronicgyroscope, electronic accelerometer, optical tracking, combinationsthereof, and the like. Beneficially, the firearm 104 may continue totrack position of the user through alternate mechanisms that avoidreliance on signals from sources external to the network-blockedlocation, and may report this positioning to external devices throughthe network. In embodiments, the controller of the network-blockeddevice may adjust polling of the non-wireless mechanisms to providetracking at suitable granularity. For example, the granularity mayincrease or decrease based on the presence or strength of the externalsignal source. In embodiments, the non-wireless movement sensors arepolled at a first rate and, in response to the strength of the externalsignal source being at or below a predetermined threshold, are polled ata second rate that is higher than the first rate. The second rate isselected such that movements of the network-blocked device can berecorded and analyzed to determine a geospatial position of the user.

In embodiments, the polling rate is dynamically adjusted based, withoutlimitation, on immediately prior movements of the user or thenetwork-blocked device. For example, to maintain a desired level ofgranularity, the polling rate of a running user will have to be greaterthan the polling rate of a walking user. For example, if a measurementtaken by the IMU is above a predetermined threshold, the controller mayincrease the polling rate of the sensor to thereby accurately capturethe movement of the user.

In embodiments, the firearm usage monitoring system 2800 is configuredto alert users of the firearms 104 or commanders of the users that thefirearm 104 is entering or within a network-blocked location.Beneficially, such alerting improves situational awareness of the usersand/or commanders and may, in embodiments, provide for neutralization ofthe signal-interference source or rerouting of users around the affectedlocation. For example, alerting a user of the presence of a networkblocked location provides for the user to repeatedly move locations andtest for network blocking such that a perimeter for the network-blockedlocation may be determined.

Beneficially, the firearm usage monitoring system 2800 may determine thepresence of a network-blocked location without orchestrating or alteringmovement of users. For example, the network-blocked location may bedetermined by the firearm usage monitoring system 2800 in response tolosing connection with two or more devices (e.g., firearms 104 orwearable devices). In embodiments, the firearm usage monitoring system2800 is configured to determine a network blocked location includingdetecting a loss of network communication from two or more deviceswithin the same geolocation and determining, in response to the loss ofnetwork communication from the two or more devices, a perimeter of thenetwork-blocked location, and, optionally, updating, in response toadditional devices losing network communication and/or any of the two ormore devices regaining network communication, the perimeter of thenetwork-blocked location.

In embodiments, the perimeter is determined through signal-strengthanalysis. For example, the signal strength for communications from thetwo or more devices immediately preceding the loss of networkcommunication may be determined and compared to models to determineinterference sources. The signal strength may be determined via, forexample, packet loss or other indicia of connectivity for the two ormore devices.

In response to the signal strength diminishing in a pattern matching afirst model, the firearm usage monitoring system 2800 determines thatthe interference is incidental and may geofence corresponding areas asnetwork-blocked locations. For example, the firearm usage monitoringsystem 2800 may correlate the incidental interference to stored datasuch as schematics, maps, images, or video of structures, bystanderdensity, terrain, combinations thereof, and the like to determineperimeters of the network-blocked locations. In response to the signalstrength diminishing in a pattern matching a second model, the firearmusage monitoring system 2800 determines that the interference is frompassive-denial mechanisms and geofences corresponding areas asnetwork-blocked locations. For example, the firearm usage monitoringsystem 2800 may correlate the active interference to stored data, asdescribed above, to determine perimeters of the network-blockedlocations. For example, the firearm usage monitoring system 2800 maydetect a rapid diminishment of signal from a user on opposite sides of aknown structural element (e.g., entering a building) and determine thatthe building is considered a network-blocked location.

In response to the signal strength diminishing in a pattern matching athird model and/or a fourth model, the firearm usage monitoring system2800 determines that the interference is from active-denial mechanisms,geofences corresponding areas as network-blocked locations, and,optionally, determining the location of the active-denial mechanisms.For example, the firearm usage monitoring system 2800 may use a thirdmodel to detect a falloff generally following the inverse square lawfrom a shared point (e.g., the signal strength of the two or moredevices being generally equal at a given radius) and determine thelocation of the active-denial mechanism at the convergence point fromthe two or more network-blocked devices. Additionally, or alternatively,the firearm usage monitoring system 2800 may use a fourth model todetect simultaneous signal dropping from geospatially proximate users,eliminate false-positive detection from, for example, loss of a networknode, and determine an area of the active-denial mechanism.Beneficially, this area may be reduced to a location by directingactions of non-blocked users to test the perimeter of thenetwork-blocked location.

The cloud-compute-constrained state is configured to inhibit loss offidelity in situational awareness for commanders and/or users of thefirearm usage monitoring system 2800. In embodiments, the firearm usagemonitoring system 2800, when operating in the cloud-compute-constrainedstate, is configured to push data processing for lower-priority datatoward the edge of the network so that server resources are allocated,dedicated, or otherwise available to process the highest-priority data.For example, if the firearm usage monitoring system 2800 is connected tofirearms 104 within a unit engaged in a firefight and to firearms 104within a unit patrolling a secured area, the firearm usage monitoringsystem 2800 may dedicate server-compute resources to processing datacollected by the engaged unit while edge-compute resources (e.g.,firearms 104, wearable devices, stationary devices, and connectionpoint) process data collected by devices of the patrol unit.Beneficially, the firearm usage monitoring system 2800 may also reducenetwork traffic by pushing data processing of lower-priority data to thedevices collecting the data such that transmission of thehigher-priority data is uninhibited by interfering signals or trafficcongestion.

In embodiments, the controller is configured to detect aspects, thecloud-compute-constrained state and switch, in response to detecting thecloud-compute-constrained state, the firearm 104 to acloud-compute-constrained state including one or more of storingacquired data to non-volatile memory on the firearm, processing acquireddata to reduce overall payload for delivery to the remote server, andadjusting communication intervals to optimize packet size for processing(e.g., sending larger packets to reduce processing overhead) similar tothose processes discussed above. Additionally, or alternatively, thecloud-compute-constrained state may include one or more of storingacquired data to non-volatile memory on peer devices and processing, viapeer devices, data acquired by the firearm 104.

The situational states are configured to provide prioritizationinformation for data collected by the firearms 104. Additionally, oralternatively, the situational states may provide tags to data formachine learning applications to optimize model training. Thesituational states may include, for example, a training state, adeployed state, and an engaged state. The situational states may bedetermined via, for example, user actuation with the firearm usagemonitoring system 2800, detection of the firearm 104 within a geofencedarea, receiving signals from one or more beacons or wireless devices,duty schedule, receipt of messages from a remote server, activation ofconnected devices (e.g., a siren or body camera), an inertialmeasurement exceeding a predetermined threshold (e.g., force requiredduring unholstering), prolonged sub-threshold actuation (e.g., holsteredfirearm while user is running), actuation of grip sensors combinationsthereof, and the like.

The controller may operate the firearms 104 in the training state whenusers are engaged in a training exercise. For example, locations such asfiring ranges and training fields may be outfitted with beacons suchthat firearms 104 entering the signal range of the beacons operate inthe training mode and data collected by the firearms 104 may be labeledas training-mode data. In embodiments, the firearms 104 connect to asecondary server when operating in the training mode to transmit, store,and/or process collected data. The secondary server may include, forexample, an on-site device such as a local server or deployable device.Beneficially, the training state may provide optimized network trafficand/or reduced cost operation by pushing data storage and processingtoward the network edge and/or connecting to a secondary server that isseparate from the remote server used, for example, during operation inthe engaged state. Data collected while operating in the training statemay be accessed later via an API or other data communication interface.

The controller may operate the firearms 104 in a deployed state whenusers are deployed to locations where the firearms 104 may be used butare not currently engaged. For example, a firearm 104 may operate in thedeployed state when a user is on-duty and outside of a trusted location(e.g., an on-duty officer outside of the station or patrol vehicle).Beneficially, the deployed state may provide optimized resourceallocation of the remote server by allowing for higher latency intransfer and processing of data collected by a firearm 104 in thedeployed state.

The controller may operate the firearm 104 in an engaged state when thefirearm 104 has been fired or when a firing event is likely. Forexample, the firearm 104 may operate in the engaged state in response todetecting an unholstering event, detecting an aiming action of thefirearm 104, engagement of peers that are proximate to the firearm 104,detection of shots fired by another firearm, receipt of a message sentby the remote server, combinations thereof, and the like. Inembodiments, the controller operates in an engaged state only when thedetected event occurs outside of a trusted location (e.g., unholsteringwithin a police station will not trigger the engaged state unlessaccompanied by manual actuation of the engaged state or a firing event).Beneficially, the engaged state provides for collection and transfer ofhigh-fidelity situational data.

While the discussion of detection of conditions, detection ofconditions, operation states, and corresponding mitigation or operationhave been discussed with reference to the firearm 104, it should berecognized that such discussion may be applied to other components ofthe firearm usage monitoring system 2800, such as connection points,wearable devices, stationary devices, etc.

In embodiments, the firearm usage monitoring system 2800 providesprocessed data to third-party software, such as a geospatialinfrastructure and military situational awareness app, for display. Forexample, the firearm usage monitoring system 2800 may provide firingdata, target locations, engagements, aiming cones, firing cones, etc. tothird-party software, such as Android Tactical Assault Kit (ATAK),Android Team Awareness Kit (ATAK), Digitally Aided Close Air Support(DACAS), Safe Strike, and other situational awareness software orapplications for military, law enforcement, and first responders.

The firearm usage monitoring system 2800 may display to firearm users orcommanders projected, collected, and/or analyzed information related to,without limitation, threats, team members, friendly units, bystanders ornon-threatening persons, images or videos captured by users, subordinateor commanding units, objective, VIP or person of interest, exit routes,vigilance or potential threat level, combinations thereof, and the like.Threat information includes, without limitation, threat location, threatmovement, threat field of view, threat firing cones, threat-controlledareas, and threat-viewable areas. Team member information includes,without limitation, team member position, team member movement, teammember field of view, team member firing cones, team member aimingcones, team member ammunition status, team member mobility status, andteam member support needs. Friendly unit information includes, withoutlimitation, friendly-unit positions, friendly-unit movement,friendly-unit firing cones, friendly-unit aiming cones, friendly-unitsupport needs, and friendly-unit status. Information related tobystanders or non-threatening persons includes, without limitation,position, number, and/or danger to the bystanders. Images or videoscaptured by users includes, without limitation, images or video capturedby the firearm 104, devices coupled thereto, connected devices, orsupport devices. Subordinate or commanding unit information includes,without limitation, position, grouping, number, ammunition, status, andsupport needs. Information related to objectives includes, withoutlimitation, location, routing information, time to completion orinitiation of action, and information regarding related objectives evenif a user or commander is not responsible for the objective. Informationrelated to VIP or person of interest includes, without limitation,position, exposure, movement, routing, status, and coverage. Exit routeinformation includes, without limitation, pathing, exposure, coverage,and alternate routes. Vigilance or potential threat level informationincludes, without limitation, status based on substantially real-timeinformation such as proximity to engaged units, knowledge of threats,and actions of team members or friendly units (e.g., unholstering oraiming).

The firearm usage monitoring system 2800 may communicate the informationthrough a flat display or a tiered display. The information may bedisplayed automatically as an alert, automatically as a view-change, asa prompt that awaits user interaction, and in response to receipt ofuser input.

For example, the firearm usage monitoring system 2800 may display to acommander an initial view displaying a plurality of users grouped byunits and, in response to detecting a triggering event, initiate a viewchange that communicates information related to the triggering event,initiate an alert that provides information related to the triggeringevent, or initiate a prompt that awaits input from the commander toinitiate the view change.

The view changes may include, without limitation, changing the displayedlocation or changing the displayed tier. In embodiments, the view changeincludes zooming in on the position of the device detecting thetriggering event. In embodiments, the view change includes changing thedisplayed indicia related to the device detecting the triggering event.More particularly, the displayed indicia may include an animated,alternating transition between the initial indicia and the alertindicia. The animation may continue for a predetermined period of timeor await interaction with the display. In embodiments, the view changeincludes transitioning from displaying the units in a group todisplaying the units as ungrouped in response to the triggering eventoccurring to a user within the group.

Events that trigger communicating the information through an alert, viewchange, or prompt include, for example, detection of grip events,unholstering, movement of user with firearm 104 such as repositioning(e.g., walking or intermittent running) or pursuit (e.g., prolongedrunning), aiming, firing, seeking cover (e.g., detection of crouching orrotational movement of user as measured by the IMU), low ammunition,weapon malfunction, or communication loss. In embodiments, thetriggering events require the detected event to be repeated by two ormore users within a group (such as an operational unit). In embodiments,the triggering events require the detected event to be repeated inproximity to another event. The proximity may be, for example,geographical proximity, temporal proximity, within display grouping(e.g., the current display shows multiple users as a single icon orindicia), within same peer group, or within same unit (e.g., by a memberof the unit and subordinates or commanders). For example, detection of agrip event by two users within 20 feet of each other triggers the actionwhereas detection of a grip event by two users that are outside 300 feetaway from each other does not trigger the action. Beneficially, suchmembers optimize situational awareness by inhibiting false-positivedisplay changes.

Grip events may be determined through suitable algorithms and mechanismssuch as grip detection, holster status, movement of unholstered weapon,and aiming of the firearm 104. For example, grip detection may includeuse of an inertial measurement unit, capacitive member, electric fieldsensor, inductive sensor, shadow detection, infrared detection,conductivity sensors, resistive sensors, and other suitabletechnologies. In embodiments, a hand of the user will incidentallyinteract with the grip-detecting mechanism when the firearm 104 isgrasped. For example, a sensor coupled to the trigger guard may detectcontact of the user with the trigger guard (e.g., an aimed position) anda conductivity sensor coupled to the trigger to detect contact betweenthe user with the trigger (e.g., a firing position).

Holster status may be determined by, for example, use of hall effectsensors, eddy current sensors, near-field sensors, magneto-resistivesensors, inertial measurement unit, capacitive member, electric fieldsensor, inductive sensor, shadow detection, infrared detection,conductivity sensors, resistive sensors, and other suitabletechnologies. For example, the firearm 104 may include an eddy currentsensor that is actuated in response to a grip event, is disposedproximate a conductive material of the holster when the firearm 104 isproperly seated within the holster and reacts to relative movementbetween the eddy current sensor and the conductive material.

Movement with the weapon out may be detected, for example, by usesuitable mechanisms and algorithms such as those employing an inertialmeasurement unit or optical measurements. In embodiments, the firearmusage monitoring system 2800 is configured to detect a pattern ofmovement from the IMU indicative of user movement. For example, the userslowly moving with a pistol in-hand will have a periodic oscillation ina dimension following the barrel of the weapon, the user running withthe pistol in-hand will have a more rapid oscillation that is pendulousin the frame of reference and generally orthogonal to the barrel, andthe user moving with a rifle in hand will have an oscillation that is atan oblique angle to the barrel. Additionally, or alternatively, opticalmethods such as tracking objects captured between successive images maybe used to track movement or, if subsequent images return out of focus,the firearm usage monitoring system 2800 may determine that the firearm104 is moving at a rapid pace.

A firearm malfunction or ammunition state may be determined, withoutlimitation through suitable devices and algorithms that coordinate withdata collected by sensors from other devices. In embodiments, conditionsindicative of a malfunction event include, without limitation, use of asidearm by the user while having sufficient ammunition for the primaryweapon, a sustained absence of fire while proximate users continue tofire, and/or detecting an abnormal firing pattern. The abnormal firingpattern may be detected, for example, through comparison to knownpatterns for similar firearms or to previously acquired firing patternsfrom the firearm.

In embodiments, connection points 1200 and 1202 provide data storage.Connection points 1200 and 1202 gather data when a connected device isgripped through minutes after the device is disengaged. If connectionpoints 1200 and 1202 cannot transmit to server device 112 or to an edgedevice on the network (e.g., not available, out of range), it may store(e.g., for up to 30 days) in onboard memory (e.g., through high datarate memory). Once available, the system may restart the transmissionprocess, so that the data is sent over.

In embodiments, system 100 provides power management capabilities. If adevice connected within range of connection point 1200 and/or connectionpoint 1202 is in motion but not in use, a low power mode (e.g., withoccasional pinging) may be implemented to maintain general awareness ofthe location of the user. The device transmits a location every onesecond. If not used for a period of time, (e.g., for a half hour) thedevice may send one message at a defined interval, such as every second,every minute, every one-half hour, every hour, or at other intervals.

Beneficially devices of the firearm usage monitoring system can beconfigured to harvest energy from electromagnetic frequenciestransmitted by other devices. For example, the firearm 104 may include awireless-energy harvesting mechanism including a suitable circuit forharvesting the electromagnetic radiation. In embodiments, thewireless-energy harvesting mechanism includes a receiving antenna thatis configured to receive the electromagnetic radiation, a rectifier thatis configured to convert the received alternating current to directcurrent, and a DC-DC converter that is configured to alter voltage ofthe rectified current to a desired voltage. The antenna may be sharedwith the communications interface of the firearm 104.

In embodiments, the data collection rate is adjusted based on the amountof energy being harvested. Beneficially, such adjustment can provideimproved wayfinding in network-denied environments without sacrificingbattery life. For example, the increased energy output from a jammer maybe harvested to provide improved battery life in network-deniedenvironments and compensate for extra energy used in increasing datacollection rates to compensate for extra energy expended incounteracting the network denial.

Referring to FIG. 13 , connection point 116 is shown as includingnetwork interface 1300, sensors 1302, signal prioritization module 1304,and signal compression module 1306. Network interface 1300 includeshardware and/or software for establishing connections, or otherwise forallowing connections to be established, between connection point 116 anddevices within a physical range of connection point 116 and betweenconnection point 116 and server 112 and/or one or more other computingdevices used to implement the functionality of system 100. Networkinterface enables connection point 116 to connect to one or more of anetwork of computers (e.g., a LAN, a WAN, a VPN, a P2P network, or anintranet), a network of networks (e.g., the Internet), or anothernetwork (e.g., a cellular network). For example, network interface 1300can enable communications over Ethernet, TCP, IP, power linecommunication, Wi-Fi, Bluetooth®, infrared, RF, GPRS, GSM, FDMA, CDMA,EVDO, Z-Wave, ZigBee, 3G, 4G, 5G, another protocol, or a combinationthereof.

Sensors 1302 include one or more sensors used to record measurementsrelating to the use of connection point 116. Sensors 1302 may, forexample, include one or more of a geolocation sensor (e.g., forconnecting to GPS and/or other global navigation satellite systems), animage sensor, a vibration sensor, an audio sensor, an IMU, or the like.In embodiments, sensors 1302 are used to sense information about theenvironment to which connection point 116 is deployed. For example,sensors 1302 can be used to capture image, video, or audio from thatenvironment. In another example, sensors 1302 can be used to detectvibrations within that environment (e.g., caused by natural or man-madeevents). In embodiments, information collected using sensors 1302 can beused to enhance, supplement, clarify, or otherwise process signalsreceived from devices connected to connection point 116. For example,where such a signal does not include geolocation information indicatingwhere the device form which the signal originates is located, connectionpoint 116 can add such geolocation information to the signal beforetransmitting the signal to server 112. In another example, connectionpoint 116 can add timestamp information to such a signal beforetransmitting it to server 112.

Signal prioritization module 1304 prioritizes channels of communicationbetween connection point 116 and devices within a connection range ofconnection point 116. Bandwidth may be limited in the deploymentlocation, for example, due to a distance between connection point 116and a nearest network signal provider, capabilities of the connectionpoint 116 itself, and/or another constraint. However, there may at timesbe a relatively large number of devices which attempt to connect toconnection point 116. For example, where each user within the deploymentlocation has a personal computing device and at least one firearm, theremay be too many devices attempting to connect to connection point 116compared to the availability of network bandwidth made available byconnection point 116. In such an event, signal prioritization module1304 can be used to prioritize connections for certain devices. Forexample, connections may be prioritized based on a time since a lastestablished connection between a device and connection point 116, a typeof the device, a type of signal or information thereof beingcommunicated from the device, a distance between the device andconnection point 116, information associated with a user of the device(e.g., based on user rank, skill, or the like), an amount of bandwidthrequired for the connection with the device, other criteria, or acombination thereof. In embodiments, connection point 116 may beconfigured to limit the total number of devices which may connect to itat a given time.

Signal compression module 1306 compresses signals received from devicesconnected to the connection point 116, for example, to prepare thesignals for transmission to server 112. Signal compression module 1306compresses the signals to reduce a bit rate at which the signals aretransmitted over a network. In embodiments, signal compression module1306 may use lossless compression technique to compress a signal. Inembodiments, signal compression module 1306 may use lossy compressiontechnique to compress a signal. In embodiments, signal compressionmodule 1306 may use lossless/lossy hybrid compression to compress asignal, such as where a portion of a signal is processed using losslesscompression and another portion of the same signal is processed usinglossy compression. The type of compression used may be based on theinformation included in a signal. For example, information whichrequires high fidelity when reconstructed for viewing may be compressedusing lossless compression, while other information may be compressedusing lossy compression. In embodiments, connection point 116 may beconfigured to identify types of information which require high fidelityand compress those types of information using lossless compression. Inembodiments, connection point 116 may be configured to decompressedcompressed data received from server 112 or otherwise from a computingdevice located outside of the deployment location.

To further describe some embodiments in greater detail, reference isnext made to examples of techniques which may be performed by or inconnection with a firearm monitoring and remote support system, forexample, system 100. The techniques include technique 1400 of FIG. 14 ,technique 1500 of FIG. 15 , and technique 1600 of FIG. 16 . Technique1400, technique 1500, and/or technique 1600 can be executed usingcomputing devices, such as the systems, hardware, and software describedwith respect to FIGS. 1-13 . Technique 1400, technique 1500, and/ortechnique 1600 can be performed, for example, by executing amachine-readable program or other computer-executable instructions, suchas routines, instructions, programs, or other code. The steps, oroperations, of technique 1400, technique 1500, and/or technique 1600, oranother technique, method, process, or algorithm described in connectionwith the embodiments disclosed herein, can be implemented directly inhardware, firmware, software executed by hardware, circuitry, or acombination thereof. For simplicity of explanation, technique 1400,technique 1500, and/or technique 1600 are each depicted and describedherein as a series of steps or operations. However, the steps oroperations in accordance with this disclosure can occur in variousorders and/or concurrently. Additionally, other steps or operations notpresented and described herein may be used. Furthermore, not allillustrated steps or operations may be required to implement a techniquein accordance with the disclosed subject matter.

Referring to FIG. 14 , at 1402, a signal including sensor information isproduced. The sensor information is produced using one or more sensorsof a device within a deployment location. For example, the sensorinformation may be produced using one or more of a geolocation sensor,an image sensor, an IMU, or another sensor configured to recordmeasurements associated with a firearm, wearable device, stationarydevice, robot, or another device. The signal may be produced using aprocessor of the device. For example, an ASIC, FPGA, or other units mayreceive the sensor information from the sensors used to record it andproduce the signal using that sensor information.

At 1404, the signal is transmitted to a server device outside of thedeployment location. The server device runs application software forproviding remote support to users of devices (e.g., firearms) within thedeployment location. In embodiments, the device at which the signal isproduced may directly transmit the signal to the server device. Inembodiments, a connection point intermediate to the device at which thesignal is produced and to the server device may be used to communicatethe signal from the device to the server device.

At 1406, the application software uses the sensor information includedin the signal to detect a threat within the deployment location. Thethreat can be or include one or more hostile combatants or other sourcesof potential injury to person or damage to property of the users ofsystem 100. In embodiments, detecting the threat using the restoredsensor information can include processing the restored sensorinformation to detect a change in an orientation of the device at whichthe signal is produced. For example, where the device is a firearm, therestored sensor information can indicate that an orientation of thefirearm has changed from one of a gripping orientation or a drawingorientation to one of a pointing orientation or a firing orientation. Inembodiments, detecting the threat using the restored sensor informationcan include processing the restored sensor information to detect adischarge of a firearm. For example, the discharge may be detected usingone or more sensors of the firearm, a wearable device worn by a user ofthe firearm at the time of the discharge, or another device.

In embodiments, detecting the threat can include processing restoredsensor information from multiple devices. For example, sensorinformation received from two firearms can be used to detect the threat.The sensor information can be processed to determine a change inorientation of at least one of the two firearms. Cones of fire for eachof the firearms can then be updated based on the sensor information.Responsive to a determination that the cones of fire of those firearmscoalesce as a result of such updating, the coalescence can be used todetect the threat. In embodiments, the threat may be detected based onthe coalescence of the cones of fire alone or based on additionalinformation which is used to clarify the reason for the coalescence ofthe cones of fire. For example, imaging data captured using a camera orother asset within the deployment location can be used to verify whethera location at which the coalesced cones of fire are pointing includes athreat. In another example, sensor information indicating a firing ofone or both firearms associated with the coalesced cones of fire can beused to detect the threat. In yet another example, one or more users ofthe firearms associated with the coalesced firearms can indicate thepresence of a threat.

At 1408, an action to perform in response to the detected threat isdetermined. The application software can automatically determine anappropriate action to take based on the nature of the detected threatand/or based on information collected from one or more devices withinthe deployment location. In embodiments, the action to perform can bedetermined based on a severity of the detected threat. For example, ahighly severe threat may call for the deployment of a large number ofreinforcements to the deployment location, whereas a moderately orminimally severe threat may call for the deployment of fewerreinforcements to the deployment location.

At 1410, the application software causes a deployment of responseinfrastructure to perform the action. In embodiments, the applicationsoftware can cause the deployment of the response infrastructure bytransmitting a command, processed at a device local to the responseinfrastructure, to initialize the use and/or operation of the responseinfrastructure. In embodiments, the application software can cause thedeployment of the response infrastructure by indicating a recommendationfor the response infrastructure within a GUI of the applicationsoftware. For example, a remote user of the application software caninteract with the application software to approve or modify therecommendation.

Referring to FIG. 15 , at 1502, first sensor information is received.The first sensor information may be received from one or more deviceswithin a deployment location. For example, the first sensor informationmay be received from one or more firearms, wearable devices, stationarydevices, robots, or other assets. The first sensor information includesmeasurements recorded using one or more sensors of the devices. Forexample, the first sensor information may include measurements recordedusing one or more of geolocation sensors, image sensors, or IMUs. Thefirst sensor information is received within one or more signalstransmitted to a server device. For example, the one or more signals maybe transmitted from a connection point intermediate to the devices andthe server device.

At 1504, a GUI is generated based on the first sensor information. TheGUI includes a visual representation of cones of fire for each firearmassociated with the first sensor information. The cones of firerepresent positions and orientations of the firearms determined based onthe first sensor information. Each firearm may have a cone of firerepresented within the GUI. The size of the cone of fire represented inthe GUI may be based on or both of a skill of a user of the firearm or atype of the firearm. For example, a user having a higher skill level mayhave a narrower cone of fire to denote a greater expectation of accurateshooting by the user.

At 1506, after the GUI is generated, second sensor information isreceived. The second sensor information indicates a change in one orboth of the position or orientation of at least one of the firearmswithin the deployment location. For example, the second sensorinformation can indicate that the orientation of one or more firearmshas changed from a gripping orientation or a drawing orientation to apointing orientation or a firing orientation, so as to denote that thefirearm has been readied for use, such as to address a threat within thedeployment location. In some cases, the orientations of multiplefirearms may be so changed as indicated by the second sensorinformation.

At 1508, the GUI is automatically updated based on the second sensorinformation. The updating may include changing a position and/ororientation of one or more cones of fire as visually represented withinthe GUI based on the second sensor information. For example, changes inthe orientation and/or position of the firearms as indicated in thesecond sensor information can be used to update the positions andorientations represented by the cones of fire for those respectivefirearms. In this way, the visual representations of those cones of firewithin the GUI is changed.

At 1510, a determination is made that two or more cones of fire visuallyrepresented in the GUI have coalesced based on the updating from thesecond sensor information. A coalescence of cones of fire refers to asituation in which the cones of fire for two or more firearms are atleast partially overlapping. Coalescence of cones of fire occurs whenusers of associated firearms have readied those firearms for firing andare pointing those firearms at a common location within the deploymentlocation.

At 1512, a threat is detected based on the coalescence of the cones offire. The threat can be or include one or more hostile combatants orother sources of potential injury to person or damage to property of theusers of system 100. A threat may be detected based on the coalescenceof the cones of fire alone or based on additional information which isused to clarify the reason for the coalescence of the cones of fire. Forexample, imaging data captured using a camera or other asset within thedeployment location can be used to verify whether a location at whichthe coalesced cones of fire are pointing includes a threat. In anotherexample, sensor information indicating a firing of one or both firearmsassociated with the coalesced cones of fire can be used to detect thethreat. In yet another example, one or more users of the firearmsassociated with the coalesced firearms can indicate the presence of athreat. In yet another example, application software which generates andupdates the GUI can detect the threat based on the number of coalescedcones of fire, the duration of time over which the cones of fire remaincoalesced, the skill levels of the users of the firearms associated withthe coalesced cones of fire, other information which may be representedwithin the GUI, or a combination thereof.

At 1514, the GUI is automatically updated to visually represent thedetected threat. The threat may be represented using an icon which isvisually distinct from icons used to represent the firearms or usersthereof within the GUI. Visually representing the detected threat withinthe GUI may include adding an icon within a location of the cone of firecoalescence.

Referring to FIG. 16 , at 1602, a signal including sensor information isproduced. The sensor information is produced using one or more sensorsof a device within a deployment location. For example, the sensorinformation may be produced using one or more of a geolocation sensor,an image sensor, an IMU, or another sensor configured to recordmeasurements associated with a firearm, wearable device, stationarydevice, robot, or another device. The signal may be produced using aprocessor of the device. For example, an ASIC, FPGA, or other units mayreceive the sensor information from the sensors used to record it andproduce the signal using that sensor information.

At 1604, the signal is transmitted to a connection point located withinthe deployment location. The connection point may be a device configuredto communicate signals from devices within the deployment location to aremote server which processes the signals to provide monitoring andother remote support to users of the devices. In embodiments, the signalmay be transmitted directly between the device and the connection point.Alternatively, in embodiments, a mobile tracking device associated witha user of the device at which the signal is produced may be used as anintermediary to communicate the signal between the device and theconnection point.

At 1606, the connection point compresses the signal received from thedevice. In embodiments, in which the connection point receives multiplesignals from the device, or in embodiments, in which the connectionpoint receives one or more signals from multiple devices, the connectionpoint can compress those signals into a single compressed signal.Alternatively, in such embodiments, the connection point can compressthose signals into separate compressed signals. For example, where theconnection point is configured for batch processing, the connectionpoint may organize signals received (e.g., within a time interval) intoone or more batches and compress each batch individually. In anotherexample, the connection point can batch signals based on the types ofdevices from which they are received. The compression of one or moresignals may be performed using a lossy compression technique.Alternatively, the compression of one or more signals may be performedusing a lossless compression technique. As a further alternative, thecompression of one or more signals may be performed using a hybridlossy/lossless compression technique.

At 1608, the compressed signal is transmitted from the connection pointto a server device. The server device is a remote server located outsideof the deployment location within which the connection point and thedevice used to produce the signal are located.

At 1610, application software running on the server device is used todecompress the compressed signal to restore the sensor information. Inembodiments, the application software determines how to decompress thecompressed signal based on information (e.g., compressed syntaxelements) recorded within the compressed signal. For example, one ormore bits can be encoded to a header file within the compressed signalto indicate, to the decompression functionality of the applicationsoftware, how to decompress the compressed signal.

At 1612, the application software uses the restored sensor informationto detect a threat within the deployment location. The threat can be orinclude one or more hostile combatants or other sources of potentialinjury to person or damage to property of the users of system 100. Inembodiments, detecting the threat using the restored sensor informationcan include processing the restored sensor information to detect achange in an orientation of the device at which the signal is produced.For example, where the device is a firearm, the restored sensorinformation can indicate that an orientation of the firearm has changedfrom one of a gripping orientation or a drawing orientation to one of apointing orientation or a firing orientation. In embodiments, detectingthe threat using the restored sensor information can include processingthe restored sensor information to detect a discharge of a firearm. Forexample, the discharge may be detected using one or more sensors of thefirearm, a wearable device worn by a user of the firearm at the time ofthe discharge, or another device.

In embodiments, detecting the threat can include processing restoredsensor information from multiple devices. For example, sensorinformation received from two firearms can be used to detect the threat.The sensor information can be processed to determine a change inorientation of at least one of the two firearms. Cones of fire for eachof the firearms can then be updated based on the sensor information.Responsive to a determination that the cones of fire of those firearmscoalesce as a result of such updating, the coalescence can be used todetect the threat. In embodiments, the threat may be detected based onthe coalescence of the cones of fire alone or based on additionalinformation which is used to clarify the reason for the coalescence ofthe cones of fire. For example, imaging data captured using a camera orother asset within the deployment location can be used to verify whethera location at which the coalesced cones of fire are pointing includes athreat. In another example, sensor information indicating a firing ofone or both firearms associated with the coalesced cones of fire can beused to detect the threat. In yet another example, one or more users ofthe firearms associated with the coalesced firearms can indicate thepresence of a threat.

In embodiments, the signal may be compressed at a device other than theconnection point. For example, the signal may be compressed at a mobilecomputing device used by a user of the device at which the signal isproduced. In another example, the signal may be compressed at the deviceat which the signal is produced. In such an embodiment, the connectionpoint may be used as an intermediate relay to receive the compressedsignal and forward the compressed signal to the server device.

To further describe some embodiments in greater detail, reference isnext made to examples of techniques which may be performed by or inconnection with a firearm usage monitoring system, for example, system100. The techniques include technique 3500 of FIG. 35 , technique 3600of FIG. 36 , technique 3700 of FIG. 37 , technique 3800 of FIG. 38 ,technique 3900 of FIG. 39 , technique 4000 of FIG. 40 , and technique4100 of FIG. 41 .

Technique 3500, technique 3600, technique 3700, technique 3800,technique 3900, technique 4000, and/or technique 4100 can be executedusing computing devices, such as the systems, hardware, and softwaredescribed above. Technique 3500, technique 3600, technique 3700,technique 3800, technique 3900, technique 4000, and/or technique 4100can be performed, for example, by executing a machine-readable programor other computer-executable instructions, such as routines,instructions, programs, or other code. The steps, or operations, oftechnique 3500, technique 3600, technique 3700, technique 3800,technique 3900, technique 4000, and/or technique 4100, or anothertechnique, method, process, or algorithm described in connection withthe embodiments disclosed herein, can be implemented directly inhardware, firmware, software executed by hardware, circuitry, or acombination thereof. For simplicity of explanation, technique 3500,technique 3600, technique 3700, technique 3800, technique 3900,technique 4000, and/or technique 4100 are each depicted and describedherein as a series of steps or operations. However, the steps oroperations in accordance with this disclosure can occur in variousorders and/or concurrently. Additionally, other steps or operations notpresented and described herein may be used. Furthermore, not allillustrated steps or operation may be required to implement a techniquein accordance with the disclosed subject matter.

Referring now to FIG. 35 , at 3502, a firearm operates in a first state.The firearm includes at least one sensor configured to recordinformation related to usage of the firearm, a communication interfaceconfigured to transmit data to a connected device, a buffer operativelycoupled to the at least one sensor, and a controller operatively coupledto the buffer, the communication interface, and the at least one sensor.The buffer is configured to store the information related to usage ofthe firearm. The first state includes transmitting data collected by theat least one sensor to the connected device in substantially real time.At 3504, a cloud-constrained condition is detected. Thecloud-constrained condition systematically inhibits communicationbetween the firearm and the connected device. For example, thecloud-constrained condition may be caused by a physical obstruction or anetwork-blocking action. At 3506, the firearm is operated in a secondstate in response to detecting the cloud-constrained condition. Thesecond state includes altering data transmission to maintain datafidelity. For example, data transmission intervals may be extended toreduce transfer overhead, the collected data may be processed to reducethe amount of data transmitted, the data may be stored for latertransmission, etc.

Referring now to FIG. 36 , at 3602, communications are established witha firearm. The firearm includes a plurality of sensors configured torecord information related to usage of the firearm, a communicationinterface configured to transmit data to a connected device, and acontroller operatively coupled to the communication interface and theplurality of sensors. At 3604, it is repeatedly determined whether oneor more criteria related to usage of the firearm are satisfied. Thecriteria include a first criterion and a second criterion. In someaspects, the first criterion is the firearm being holstered and thesecond criterion is selected from the group consisting of the firearmbeing geolocated within a predetermined area, movement of the firearmbeing below a predetermined threshold, movement of the firearm beingoutside of a predetermined pattern, and a user being on-duty. At 3606,the firearm is operated in a first standby state in response thecriteria being satisfied. At 3608, the firearm is switched from thefirst standby state to a second standby state in response to the firstcriterion being unsatisfied while the second criterion remainssatisfied. At 3610, the firearm is activated to a real-time-monitoringstate from either the first standby state or the second standby state inresponse to the second criteria being unsatisfied. Thereal-time-monitoring state includes substantially real-time datatransfer to the connected device of the information related to usage ofthe firearm.

Referring now to FIG. 37 , at 3702, communications are established witha firearm. The firearm includes at least one sensor configured to recordinformation related to usage of the firearm and a controller operativelycoupled to the communication interface and the at least one sensor. At3704, a communication interface configured to transmit data to aconnected device, it is repeatedly determined whether one or morecriteria related to usage of the firearm are satisfied. The criteriainclude one or more of the firearm being geolocated within apredetermined area, the firearm being holstered, movement of the firearmbeing below a predetermined threshold or outside of a predeterminedpattern, a user being on-duty, or contact of the user with apredetermined location on the firearm. At 3706, the firearm is operatedin a standby state in response to the criteria being satisfied. At 3708,the firearm is switched from the standby state to a real-time-monitoringstate in response to any of the criteria being unsatisfied. Thereal-time-monitoring state includes substantially real-time datatransfer of the information to the connected device.

Referring now to FIG. 38 , at 3802, a plurality of users is monitored.Each of the plurality of users has a respective one of a plurality offirearms. At 3804, signals are received from the plurality of firearmsregarding usage thereof. At 3806, a display device displays a graphicalrepresentation of geospatial positioning of the firearms. At 3808, acontroller determines operating states of each of the plurality offirearms. At 3810, the controller detects a change in the operatingstate of at least one of the plurality of firearms. At 3812, an updatedgraphical representation is provided in response to detecting the changein operating state. The updated graphical representation providesindicia of the change in the operating state.

Referring now to FIG. 39 , at 3902, communications are established witha connected device via a communication interface. At 3904, a jammingsignal that inhibits communication with the connected device is detectedusing the communication interface. At 3906, in response to detecting thejamming signal, communication with the connected device is stopped. At3908, a wireless-energy harvesting mechanism leaches power from thejamming signal in response to detection thereof. The wireless-energyharvesting mechanism includes a receiving antenna configured to receivethe jamming signal, a rectifier configured to convert the receivedsignal to direct current, and a DC-DC converter configured to altervoltage of the direct current to a desired voltage.

Referring now to FIG. 40 , at 4002, the battery powers components of thefirearm in a first sensing mode. At 4004, a communication interface ofthe firearm monitors electromagnetic flux proximate the firearm. At4006, a wireless-energy harvesting mechanism leaches power from theelectromagnetic radiation in response to the electromagnetic radiationdensity exceeding a predetermined threshold. The wireless-energyharvesting mechanism includes a receiving antenna configured to receivethe jamming signal, a rectifier configured to convert the receivedsignal to direct current, and a DC-DC converter configured to altervoltage of the direct current to a desired voltage. At 4008, operating,via the battery and the harvested power, the components of the firearmin a second sensing mode. The second sensing mode expends more energythan the first sensing mode.

Referring now to FIG. 41 , at 4102, sensor information is received froma firearm. At 4104, an event model is used to evaluate the sensorinformation. In embodiments, the event model is created by obtainingtimestamped information from a first sensor type coupled to a firearmand a second sensor type, selecting a plurality of monitored eventsincluding a discharge event, labeling the timestamped information withthe respective one or more of the monitored events in response to thetimestamped information occurring contemporaneously with a respectiveone or more of the monitored events, grouping items of the timestampedinformation that are sensed temporally proximate to the monitored eventby the sensor or another sensor within the sensor types with eachrespective item of labeled information, splitting the grouped data intoa first portion and a second portion, training the event model using thefirst portion via machine learning, and evaluating the event model usingthe second portion. After the event model is trained and passesevaluation, the model may be implemented in the system or componentsthereof. At 4106, the evaluation determines occurrence of one or more ofthe monitored events. At 4108, indicia communicating the occurrence ofthe one or more of the monitored events are displayed to a viewer via ahuman interface in response to determining that a monitored event willoccur.

Referring to FIG. 17 , gestures, positions and locations of a firearmindicative of or in preparation for live fire are shown. In particular,gestures and weapon orientations 1700 that can serve to as inputs and/ortriggers to system 100 are shown. In embodiments, gestures and weaponorientations 1700 can include a gripping gesture and orientation 1702, adrawing gesture and orientation 1704, a pointing gesture and orientation1706 that can be indicative of aiming the weapons, and a firing gestureand orientation 1708 that is indicative of live fire. Firing gesture andorientation 1708 can further include firing directions, angles of theweapon, rates of fire information, and the like. At each detectedgesture and orientation 1700, system 100 can, in many examples, transmita spot report 1710.

In various examples, spot report 1710 can include: unit identification,date and time information, location information, and threat/enemyactivity information. In embodiments, the unit identification canidentify user profiles, asset identities, and the like. In embodiments,the unit identification can also be used to determine what units (ordivisions thereof) of a deployed force are associated with the weapon.By way of these examples, system 100 can verify the authenticity of theunit identification, deploy encrypted communication and other securitymeasures to ensure secured connectivity with the weapon and its properpairing and continued proper pairing with the user. In embodiments, theunit identification can be associated with a soldier. In embodiments,the unit identification can be associated with a police officer. Inembodiments, the unit identification can be associated with a securityagent, a private homeowner or business owner, a unit of a corporatesecurity force, and the like.

In embodiments, spot report 1710 can include the location informationfrom GPS, inertial measurement information, other mapping information,or the like. In embodiments, the location information can also includeoverlays from location information provided by other associated users,components, network location information, and other electromagneticinformation in the vicinity. In embodiments, the location informationcan also include information from one or more attitude and headingreference systems from one more units deployed with the user. By way ofthis example, sensors on three axes can provide attitude information foraircraft, unmanned aerial vehicles, drones other deployable robots, orthe like and those sensors can supply roll, pitch, and yaw, or otherthree axes examples to enhance location information. The sensors caninclude solid-state or microelectromechanical systems gyroscopes,accelerometers, and magnetometers.

In embodiments, spot report 1710 including the threat/enemy activityinformation can include size, location, and activity for multipletargets. By way of this example, threat/enemy activity can be determinedor can be made more confident with information obtained by adjacentassets in the field. In many examples, unmanned aerial vehicles, drones,and the like may provide video overlays from the vicinity to confirmthreat/enemy activity, contribute to the calculation of threat/enemyactivity, and to increase confidence in reporting of threat/enemyactivity and location.

In embodiments, spot report 1710 can include provisioning informationabout the weapon and current ammunition status. By way of theseexamples, the spot report can include ammunition remaining, ammunitiondischarge rate, prompts for resupply, and anticipated resupply needs. Inmany examples, the weapons can be assigned a standard stock or count ofammunition. Detection of live fire can cause the firearm usage trackingsystem 800 to calculate shot consumption and predict when the weaponwill deplete its local ammunition. Resupply information can beautomatically transmitted when predetermined levels of remainingammunition are reached or are approaching quickly at a given rate offire. In many examples, the weapon may be provisioned with many roundsof ammunition and levels of remaining ammunition can be included in spotreports. Once remaining ammunition levels dip below preset thresholds,resupply alerts can be sent. In addition, predictions can be presentedto the user to describe when ammunition will be exhausted especially ifthere is currently or recently a relatively high rate of fire.

Referring to FIG. 18 , a visualization of multiple users and assetsengaged in live fire showing spot report information includingcommunication statuses, unit identifiers, day and time information,location information, and assessments of enemy activity and locationincluding confidence indicators of threat assessment is shown. Inparticular, a third-person visualization 1800 including a street view1802 that shows multiple users 1804 and assets 1806 engaged in live fireis shown. In embodiments, third-person visualization 1800 includes spotreport information 1808 that can include communication statuses 1810,unit identifiers 1812, day and time information 1814, locationinformation 1816, and assessments of enemy activity and location 1818including confidence indicators of that assessment.

Referring to FIG. 19 , a visualization of multiple users and assetsengaged in live fire showing spot report information including unitidentifiers, location information, ammunition remaining, ammunitiondischarge rate, prompts for resupply, and anticipated resupply needs areshown. In particular, a third-person visualization 1900 including anarea view 1902 that shows multiple users 1904 and assets 1906 engaged inlive fire is shown. In embodiments, third-person visualization 1500includes spot report information 1908. In embodiments, spot reportinformation 1908 includes unit identifiers 1910, location information1912, ammunition remaining 1914, ammunition discharge rate 1916, promptsfor resupply 1918, and anticipated resupply needs 1920. In embodiments,third-person visualization 1900 includes assessments 1922 of enemyactivity and location. In embodiments, assessments 1922 of enemyactivity and location can include confidence indicators 1924 of thatassessment.

In embodiments, FIGS. 20, 21, 22, and 23 depict, embodiments of firearmusage monitoring system includes circuit board 2010 electrically coupledto battery 2012 with connecting wire 2022. The battery 2012 iselectrically coupled to entry point 2014. The entry point 2014 isconfigured to receive a hardwire connection for either electrical poweror data. The battery 2012 is mounted into first grip panel 2016. Thecircuit board 2010 is mounted into second grip panel 2018. The firstgrip panel 2016 can be joined to second grip panel 2018 on firearm 2020to form grip 2024. The grip 2024 can contain magazine 2028 that cancontain rounds 3030. The trigger 2032 can be pulled after safety 2034 isreleased to fire one of the rounds 2030 with firearm 2020.

Turning to FIG. 25 , the circuit board 2010 can be designed at a highlevel with functionality to promote extended battery life and facilitatemore detailed data recording. The entry point 2014 can be configured asa data connection point and, in this example, is shown here as a mini-Buniversal service bus (USB) connector 2100, when direct connection isapplicable. When connected to a USB cable this is a hard-wired data andpower connection 2102. The mini-B USB connector 2100 is electricallycoupled to a USB to serial universal asynchronous receiver/transmitter(UART) controller 104. This UART to USB controller 2104 comprises anintegrated modem with up to 3M Baud, a virtual communications (COM)port, and a +3.3 V level converter that operates on 8 mA or so. Forinstance, the FT231X integrated circuit meets these specifications. Ineffect, the UART to USB controller 2104 provides functionality to updatefirmware in the remainder of the system providing for substantiallygreater upgrades and improvements than other devices in this field. Inexamples, the UART to USB controller 2104 can be electrically coupled toa transmitter/receiver status light emitting diode (LED) 110 that canindicate if a firmware update is occurring.

In examples, force sensor 2120 can be electrically coupled to a firstgeneral purpose input/output pin GPIO 1 2122. The force sensor 2120 canbe a resistive based force sensor with a voltage divider for analoginput. In these examples, the force sensor 2120 will typically draw lessthan 1 mA of current from the UART to USB controller 2104. When force isimparted on the force sensor 2120, the circuit board 2010 can wake upand begin to operate (or operate beyond minimal operation). The forcesensor 2120 can be a force-sensing resistor. For instance, the FSR 2400single zone force-sensing resistor can meet these requirements.

In examples, the UART to USB controller 2104 can be electrically coupledto a Bluetooth/uC Module 2130. The bluetooth/uC Module 2130 can beconfigured to send data to and receive data from the UART to USBcontroller 2104. In some embodiments, Bluetooth/uC Module 2130 can be anRFduino stand-alone board that can be configured with an ARM Cortexprocessor and Bluetooth Low-Energy 4.0 built-in. In such examples, thiswould typically consume 20 mA peak and 9 mA normal. It is equallypossible, that the Bluetooth/uC Module 2130 can include two modules: amicroprocessor and a communication circuit which can be separated. Whilea Bluetooth communication circuit may be the easiest way to transmitdata, data can also be transmitted through the mini-B USB connector2100. Further, there is any number of possible wireless communicationsystems that could be used such as radio frequency, Wi-Fi, near fieldcommunication and other forms of electromagnetic or wired communication.

In some embodiments, the Master Out Serial In (MOSI) pin GPIO 2 2132 onthe Bluetooth/uC Module 2130, the Data Clock (SCK) pin GPIO 4 2134, theMaster In Serial Out (MISO) pin GPIO 3 2138, and the CS-MPU pin GPIO 52140 are electrically coupled to the nine-axis motion monitor 2142. Thenine-axis motion monitor 2142 may, for example, be an IMU. The nine-axismotion monitor 2142 is configured to measure and transmit data about allof the positioning of the circuit board 2010 while in motion of anykind. In many examples, this can include a Tri-axis gyro up to 2000 dps,tri-axis accelerometer up to 16g, a tri-axis compass up to 4800 uT, andprogrammable interrupt. This would typically consume 4 mA. For instance,the MPU-9250 provides this functionality. In many examples, thistriparate functionality to monitors exact orientation and track wherethe firearm travels in terms of rotation, speed, and direction. In somecases, the tri-axis compass can be accomplished with a magnetometer.Recoil and/or shot count resulting from firearm discharge can beidentified from the gathered data.

MISO pin GPIO 3 2138, SCK pin GPIO 4 2134 and MOSI pin GPIO 2 2132 arefurther electrically coupled to serial flash memory 2150. In manyexamples, the serial flash memory 2150 can operate in double transferrate or DTR mode in some cases a gigabyte of memory formed by 256 MBdie, with 100,000 erase cycles per sector. In these examples, such anarrangement can draw 6 mA. The serial flash memory 2150 can be furtherelectrically coupled to CS-Flash pin GPIO 6 2152 on the Bluetooth/uCModule 2130. In these examples, the N25Q00AA flash memory meets thisrequirement.

MISO pin GPIO 3 2138, SCK pin GPIO 4 2134 and MOSI pin GPIO 2 2132 canbe further electrically coupled to a GPS Module 2160. The GPS Module2160 is further electrically coupled to CS-GPS pin GPIO 7 162 on theBluetooth/uC Module 2130. The GPS module 2160 can be configured todetermine position within 2.5 meters of accuracy with a 10 Hz updaterate, internal real time clock, onboard read only memory, and −167 dBmsensitivity. In these examples, this can operate continuously with adraw of 30 mA continuous and 7 mA while in power save mode (1 Hz). Forinstance, The U-BLOX™ CAM-M8Q chip antenna module can meet thisrequirement. There are a lot of other kinds of GPS systems that could beequally acceptable including Glonass™, Beidou™, etc.

In some embodiments, the mini-B USB connector 100 is electricallycoupled to the UART to USB controller 2104 for sending data D+ andreceiving data D−, however, it need not operate on that voltage.Accordingly, circuit 2010 can be configured to have a system that bothrapidly charges the battery 2012 and permits data exchange. In theseexamples, the mini-B USB connector is electrically coupled to a batterycharger 2166. The battery charger 2166 is electrically coupled tobattery 2012 with a switch 2168. The battery charger can be set to 500mA and include a sense current, reverse discharge protection, andautomatically power down. For instance, charger MCP73831 can meet theserequirements.

FIG. 25 depicts embodiments with a lithium polymer battery, but otherkinds of batteries can be used as well. One battery 2012 can provide 3.7V and have an 850 mAh capacity. The battery 2012 can be electricallycoupled to a low dropout (LDO) regulator 2170. The LDO regulator 2170can step down the voltage from 3.7 V to 3.3V to provide power at avoltage that can be used by the UART to USB controller 2104 and theBluetooth/uC Module 2130. The LDO regulator 170 can be configured toprovide 300 mA output, 270 mV dropout, output fixed at 3.3 V, reversebattery protection with no reverse current, and overcurrent protection.For instance, LDO regulator LT1962 can meet these requirements. In theseexamples, the GPS module would typically operate at 3.7 V.

FIG. 24 provides examples of connecting these components. The batteryconnection PI 2172 provides a battery voltage and is attached to groundand the switch SI 2174 toggles whether the battery voltage is sent tothe rest of the system. The battery charger U3 2178 is connected to thebattery 2012, and a voltage source and, when charging engages LED C22180. The LDO regulator U6 2182 can drop the battery voltage to 3.3 V.The Mini-B USB connection J1 2184 can be joined for data purposes toUART to USB circuit U1 2188. The UART to USB Circuit U1 2188 can receivedata from Bluetooth uC/Module U4 2190, which can receive data from thenine-axis motion monitor U7 2192, serial flash memory U5 2194 and theGPS Module U2 198.

FIG. 26 depicts embodiments of an electronic system 2200 that may takethe form of a computer, phone, PDA, or any other sort of electronicdevice. The electronic system 2200 can include various types of computerreadable media and interfaces to read and write to various other typesof computer readable media. The electronic system 2200 can include a bus2205, processing unit(s) 2210, a system memory 2215, a read-only 2220, apermanent storage device 2225, input devices 2230, output devices 2235,and a network 2240.

FIG. 27 depicts embodiments of components for a firearm usage monitoringsystem 2800 including an IMU including gyro/accelerometer 2802, GPS2804, force connector 2808, power input 2810, battery charger 2812,laser 2814, regulator 2818, USB connector 2820, flash memory 2822,Bluetooth™ 2824, programmable hardware 2828, and the like.

FIG. 28 depicts embodiments of the firearm usage monitoring system 2800integrated into a grip 2900 of a weapon 2902. A circuit 2908 boardhaving one or more of the combinations of the components illustrated inFIG. 27 can be disposed within the grip 2900 of the weapon 2902 and canbe integrated so that it is almost invisible to the user other than thepresence of USB ports 904 that can be covered by the hand of the userwhen the weapon is gripped or can be omitted altogether in someembodiments.

With reference to FIG. 26 , the bus 2205 can collectively represent allsystem, peripheral, and chipset buses that communicatively connect thenumerous internal devices of the electronic system 2200. For instance,the bus 2205 can communicatively connect the processing unit(s) 2210with the read-only memory 2220, the system memory 2215, and thepermanent storage device 2225. From these various memory units, theprocessing unit(s) 2210 can retrieve instructions to execute and data toprocess in order to execute the many processes disclosed herein. Theprocessing unit(s) may be a single processor or a multi-core processorin different embodiments.

In embodiments, the bus 2205 also connects to the input and outputdevices 2230 and 2235. The input devices 2230 can enable the person tocommunicate information and select commands to the electronic system2200. The input devices 2230 can include alphanumeric keyboards,pointing devices “cursor control devices”, and the like. The outputdevices 2235 can display image generated by the electronic system 2200.The output devices 2235 can include various printers, display devicesand touchscreens that can function as both input and output devices.

The bus 2205 also couple the electronic system 2200 to the network 2240through a network adapter. In this manner, the computer can be a part ofa network of computers (such as a local area network (“LAN”), a widearea network (“WAN”), or an intranet), or a network of networks (such asthe Internet).

These functions described above can be implemented in digital electroniccircuitry, in computer software, firmware or hardware. The techniquescan be implemented using one or more computer program products.Programmable processors and computers can be packaged or included inmobile devices. The processes may be performed by one or moreprogrammable processors and by one or more set of programmable logiccircuitry. General and special purpose computing and storage devices canbe interconnected through communication networks. Some embodimentsinclude electronic components, such as microprocessors, storage andmemory that store computer program instructions in a machine-readable orcomputer-readable medium (alternatively referred to as computer-readablestorage media, machine-readable media, or machine-readable storagemedia). The computer-readable media may store a computer program that isexecutable by at least one processing unit and includes sets ofinstructions for performing various operations. Examples of computerprograms or computer code include machine code, such as is produced by acompiler, and files including higher-level code that are executed by acomputer, an electronic component, or a microprocessor using aninterpreter.

With reference to FIGS. 25 and 28 , the hardware and software, inembodiments, can be activated using one or more of any form of user feedsensor 2840, force sensor 2842, wireless remote 2844, remote on/offswitch 2848, and the like. Moreover, the hardware and software can beactivated using one or more mobile device 2850, user wearables 2852,dedicated hardware token 2854 making a wireless or wired connection, orthe like. In embodiments, the firearm usage monitoring system 2800 mayoperate with the following instructions: receiving a signal from a forcesensor 2842 such as the force sensor 2120 (FIG. 25 ). If the signal ispresent, then the firearm usage monitoring system 2800 can engage, orthe system 2800 can remain in a dormant or sleep mode with a low voltagedraw as described herein. If the signal of the force sensor 2842 is on,then the Bluetooth UC/Module 2130 can receive a signal from the GPSmodule 2160 as to where the system 2800 is presently located. As notedabove, one or more signals including those from the force sensor 2120,2842 can activate the system 2800. Once the system 2800 is active, theIMU 802 (FIG. 27 ) can provide information as to how the firearm 2020 isoriented and moved in 3D space until, in some embodiments, pressurereleased on the grip 2024. The system 2800 can determine whether thefirearm 2020 has been motionless for a preselected period, or theinformation is specifically queried. Information as to how the firearm2020 is oriented and moved in 3D space can include analyzing the firearm2020 for recoil and/or shot count when fired to discern orientation,direction, and position at the time of discharge. In examples, this datacan be stored in the flash memory 2150 and can be transmitted throughthe Bluetooth uC/Module 2130 to another Bluetooth compatible device. Theinformation including orientation, direction, and position can be alsotransmitted from the firearm 2020 at preselected time intervals,specific times, distances from certain locations (e.g., pre-definedgeo-fencing locations or distances), at the time of discharge, at thetime of reload of rounds 2030, when the safety 2034 (FIG. 21 ) isremoved, and the like.

In embodiments, the firearm usage monitoring system 2800 may record themotion of the firearm 2020 and provide geolocation information 2858,which may be coordinated with other information, such as disclosedherein. In embodiments, the system 2800 may transmit data via thenetwork connection 2240 (FIG. 26 ), such as a cellular network, to aremote server, which may be a secure server, or other remote processingcomponents, such as the mobile device 2850, cloud platform 2860, or thelike. In embodiments, the system 2800 may include efficient architectureand components for low power consumption including energy harvestingmechanisms 2862. In examples, the system 2800 can harvest the energy ofmotion of the firearm or energy from the recoil to provide power forstorage and/or reporting of data. In embodiments, methods and systemsprovide rapid, efficient determination of location. The energyharvesting mechanisms 2862 may also be configured to harvest localenergy in the radio frequency (RF) domain or other appropriate localelectromagnetic signals of sufficient strength.

In embodiments, the network connection 2240 (FIG. 26 ) by which thesystem may communicate data may be a mesh network connection 2864. Withreference to FIG. 30 , the mesh network connection 2864 may be aconnection to one or more other firearms or one or more other devices,such as a mobile robot 2868, an infrastructure device 2870, or the like.The mesh networking connection 2864 may form part of a large meshnetwork, allowing devices, such as firearms and mobile robots, tocommunicate directly with one another, rather than having to firstconnect through a centralized network communication hub, or as asupplement to communication by one or more devices to such a hub. Suchdevices may include self-disposing devices 2872, for example,self-disposing mobile robots. In embodiments, the mesh network 2864 maybe a self-organizing and fluid mesh network that organizes andreorganizes itself based on specified data, including data filtered orweighted based on specified criteria, and/or the dynamic detection ofother devices, for example with a geographic perimeter. Other devicesmay include deployable mesh network hubs 2872, also known as “pucks”,beacons, wireless access points, such as Wi-Fi access points, lightingsystems, cameras, and the like. The mesh network 2864 may also includeasset management systems, crowdsourced communications, frequencyscanning networking, cellular mesh networking or other systems. Inembodiments, devices on the mesh network 2864 may adjust locationinformation based on the relative movement of each other within the meshnetwork 2864. In embodiments, the relative movement of devices may bereported by other devices within the mesh network 2864 over the meshnetwork 2864, such as to the self-disposing devices 2872. The relativemovement of other devices may also be derived from IMUs disposed withthe other devices within the mesh network 2864. Relative movementinformation may include speed, velocity, acceleration or positioninformation, and/or event identification information 2874. Suchinformation may include threat identification information, shot accuracyinformation and the like. Event identification information may includeweapon information, information indicating a person is in anunauthorized area, soldier maneuver information (e.g., speed, direction,activity, or the like), in-position information (such as for anindividual or a device), rate-of-fire information, alternating fireinformation, maintenance required information, stoppage eventinformation, ammunition expenditure information, fight or struggleinformation and the like. In embodiments, authentication information maybe received from radio frequency identification (RFID) implants, forexample, implanted in the person. In embodiments, the relative movement,such as among devices in the mesh network 2864 like firearms 2020 andother equipment may be provided relative to at least one geographiclocation, such as through the use of data from the IMUs or from one ormore other data sources. In embodiments, location may relate to relativelocations of one or more other firearms or other devices connected tothe mesh network 2864, such as the distance, direction, and/or movementof one or more other firearms 2020 or other devices relative to a givenone. In such embodiments, geographic location and movement information2858, whether relating to a location or to another firearm or otherdevice may be communicated to a given firearm or other systems of anindividual handling a firearm over the mesh network 2864. Inembodiments, the geographic location may be an underground geographiclocation, where other geographic location detecting signals, such as GPSare not available. In embodiments, a combination of geographic locationand relative location may be understood by the system, such as where atleast one member of a mesh network has a detectable location (such as byGPS signal) and other members have locations that are determinedrelative to the known member, such as by detecting motion through theIMU 2802 or other non-GPS systems. It may be appreciated from theseembodiments that using data from the IMU 2802 on the mesh network 2864may allow the firearm usage monitoring system 2800 to provide dischargelocation information in geographic locations that may not otherwise becovered by geographic location detecting signals.

In embodiments, the mesh network 2864 connection may be a wireless meshnetwork connection and may be configured based on radio communicationfrequencies. In some situations, radio communication frequencies may besubject to interference or jamming, either intentionally or otherwise,making communication difficult or impossible when attempting toestablish a connection over the compromised frequency. Interference orjamming may include radio frequency interference or jamming, opticaljamming, noise, and the like. Because of the risk of jamming, andbecause communication reliability may be critical for user of thefirearm usage monitoring system 2800, the firearm usage monitoringsystem 2800 may detect such jamming of one or more frequencies andautomatically adjust the frequency of the mesh network 2864 to avoidusing the compromised frequency, such as by selecting a frequency notcurrently subject to interference or jamming. The firearm usagemonitoring system 2800 may then establish a wireless mesh networkconnection with another device using the selected frequency. Jamming orinterference detection may include detecting attempted signalinterception and scrambling transmitted information to avoid thedetected signal interception.

In embodiments, the firearm usage monitoring system 2800 may determinedischarge information 2878 related to the firing of the firearm 2020connected to the mesh network 2864. The discharge information 2878 mayinclude discharge location, direction of the discharge, a motion path ofthe firearm preceding discharge and/or orientation of the firearm atdischarge. Orientation information 2880 may be provided by the IMU 2802and may include enemy area location and size information, unsafe actinformation, line of fire information, shift fire information, sectorsof fire information, interlocking fire information, 360 degree perimetersecurity information and the like. The discharge information 2878 may bedetermined from motion and location information, such as provided bydevices connected to the mesh network. For example, the dischargelocation may be determined from geographic location data of one or morefirearms connected to the mesh network 2864 and may use relativemovement data provided by the other devices connected to the meshnetwork 2864, for example by analyzing relative movement data that isbased on resident IMU data from other firearms connected to the meshnetwork 2864. In embodiments, methods, systems and components areprovided for a small-footprint firearms tracking system 2882, such asone of the dimensions less than 25 mm×25 mm×4.55 mm). In embodiments,the firearm tracking system 2882 may identify movements and actionswhile in sleep mode such as to trigger transmission of alert codes. Inembodiments, the firearm tracking system 2882 may be adapted forintegration with various gun platforms, such as to interface withdifferent grips, handles, and other internal and external firearmcomponents and accessories, including being integrated entirely into thegrip of the firearm. In embodiments, the system may use over-the-airupdates, may act as or integrate with a beacon 2884, such as a BLEBeacon, which may be charged by wireless charging and may record data(such as IMU data) when in the active or inactive mode (such as to flashmemory) and may enable a sleep/hibernation mode. In embodiments,components are provided for a small-footprint firearms tracking system2882 may include Simblee (Bluetooth Low Energy, Microcontroller Unit),Micron N25Q256A13EF840E (256 Mbit Flash Memory), MPU9250 (9 axisaccelerometer, gyroscope, and magnetometer IMU), ORG1411-PM04 (OriginGPS Nano Hornet, 2.7 V), FSR-400 (Force Sensor), 800 mAh LiPo Battery,Battery Charger (MCP73831), 2.7 V Regulator (MIC5365), 3 V Laser, and/orUB-MC5BR3 (Waterproof USB connector).

In embodiments, the system may function in active modes, sleep modesand/or hibernation modes. In the active mode, the device may be in fullpower mode, such as using power for collecting readings from the IMU andGPS and transmitting them via a local protocol like BLE to an edgedevice. The laser module 2814 may also be activated. In embodiments,data can be sent in this format at relatively high data rates, such asat 30 messages/second, 50 messages/second, 100 messages/second, or thelike. A sample string may includeAB-FC-22-CC-B3-00-00-00-00-00-00-00-00-00-00-00-00-5E-89-5A-C0-71-3E-E6-C0-FA-18-9C-00-00-20-75-3F-00-80-52-3E-00-00-19-3E-00-00-64-40-67-66-00-C1-34-33-6B-00-01-BA. The guide may be as follows: AB (header), FC-22-CC-B3-00 (millisecondtimestamp), 00-00-00-00 (latitude), 00-00-00-00 (longitude), 00-00(altitude in meters), 00 (horizontal accuracy in meters), 5E-89-5A-C0(gyro x), 71-3E-E6-C0 (gyro y), FA-18-9C-C0 (gyro z), 00-20-75-3F (accelx), 00-80-52-3E (accel y), 00-00-19-3E (accel z), 00-00-B4-40 (mag x),67-66-00-CI (mag y), 34-33-6B-C0 (mag z), 01 (unit status), BA (footer).A millisecond timestamp may be used, such as in a modified Unixtimestamp, e.g., for milliseconds after 01-01-16. If BLE is unavailableor a message is not sent, this may be stored in the flash memory 2150,2822 to be sent when the device enters sleep mode. The Active mode maybe triggered when force is applied to the force sensor 2120, 2822.Depending on the configuration, the system 2800 may remain in the activemode for a specified time, such as two minutes after the force is nolonger applied, for five minutes, for ten minutes, or the like. Thistimer may be reset when force is reapplied. In embodiments, the lasermodule 2814 may be turned on at limited times, such as when the forceapplied to the force sensor (optionally based on the mode or regardlessof the mode). This mode may consume, for example, around 70 mAh ofenergy. The unit may also power down into a “sleep” mode, such as whenthere is no longer force applied to the unit and the timer has gone down(indicating expiration of active mode). In such a sleep mode, onemessage may be sent at a defined period, such as once per second, suchas containing the timestamp, location data, and current orientation data2880. The GPS module 2160, 2804 may enter an ATP (adaptive tricklepower) state where it cycles between full power and ATP to minimizepower consumption while maintaining a fix on its location. Inembodiments, a location fix may be maintained consistently, regardlessof power mode. In embodiments, the IMU may be polled at a low rate, suchas to monitor movement. If no movement is sensed for a given time, suchas five minutes, then the unit may go into another even lower powermode, referred to herein as a hibernation mode. In such as hibernationmode, the unit may continue to send messages (e.g., one per second),such as containing the timestamp, location data, and current orientationdata. The GPS module 2160, 2804 may enter hibernation where it consumes,for example, under 1 mA of power. The IMU 2802 may still be polled at alow rate. If movement exceeds a certain threshold, the unit may go intosleep mode and the GPS module 2160, 2804 may wake up to maintain alocation fix. This mode may consume, for example, under 7 mAh.

In embodiments, the firearm usage monitoring system 2800 may communicatewith external systems, such as by delivering reports, events, locationinformation, and the like. In one such embodiment, a signal may beprovided to a camera system 2880, such as a body camera worn by anindividual, to initiate recording by the camera, such as recording videoof a scene involving the individual. For example, the camera system 2888may initiate recording upon receiving a signal indicating that a weaponhas been raised into an aiming position so that the situation in whichthat activity occurred is recorded. By triggering the camera system 2888to activate one or more body cameras upon such events, use of the bodycameras may be limited to key situations, potentially reducing thestorage and data transmissions requirements for capturing, storing andtransmitting video data over networks, which can be very expensive iflarge amounts of video are captured for normal daily activities forwhich there is little use for recorded video. In these examples, theinformation obtained from the camera can be with permission and only incertain geographic zones to support privacy requirements for varioussituations. In these examples, the information obtained from the cameracan support facial recognition functionality. In further examples, theinformation obtained from the camera be of reduced quality to supportfaster capture but otherwise not support facial recognitionfunctionality or other post-processing requiring substantially highresolution. Thus, the firearm usage monitoring system 800 may enable amuch more efficient overall monitoring system, including one thatrecords video involving the user of the firearm 2020.

In embodiments, data, such as various firearm usage events (such asgripping the firearm, raising the firearm, discharging the firearm,moving around with the firearm, entering defined locations with thefirearm, and the like) may be stored, analyzed, and provided, either inraw form or in various packaged feeds, such as analytic feeds, toexternal systems. With reference to FIG. 28 , one class of system thatmay consume such data and/or analytics is an insurance system 1050,where such data may be used for various purposes, such as forunderwriting and pricing insurance contracts (such as for liabilityinsurance, accident and hazard insurance, health insurance, lifeinsurance, and others) involving one or more individuals or groups forwhom firearm-related activity is monitored by the methods and systemsdisclosed herein. This data may be used for actuarial purposes (such asto predict the likelihood of adverse events involving firearms, such asaccidents or other problems), as well as to compare the relative safetyof a given group as compared to one or more cohorts. For example, asecurity firm that wishes to obtain liability insurance can be comparedto other security firms in the same industry or area, and the extent towhich weapons are gripped, raised, or discharged can be considered indetermining whether to issue insurance and at what price insuranceshould be issued. This may include data related to on-the-job events aswell as data related to training (such as where consistent usage intraining situations may serve as a favorable indicator forunderwriting).

Methods and systems are provided herein for identifying discharges andcounting shots, discharges, etc. Conventional technologies for doing sotypically require a spring in the magazine and a system for detectingwhere the spring is positioned. For example, as another bullet went intothe chamber of the weapon, the spring position helped measure rounds ina magazine. By contrast, the present disclosure provides an externalsolid-state device that can be attached to the firearm 2020 to registerwhen one or more shots are fired. The discharge has a unique,detectable, physical profile (i.e., a discharge has recoil that has aparticular motion profile, sound profile, and the like). A recoilmeasuring system 3052 may use an IMU, including or combined withmotion-detecting/sensing elements, including one or more accelerometers,gyros, magnetometers, and the like. In embodiments, a map is developedbased on analyses of discharge events to the map the entire motionsequence caused by a typical discharge. That motion profile, which maybe unique to each weapon platform and user, can be stored and used as abasis for comparing future sensed data to determine whether a dischargeevent has occurred. Similar profiling can be used for each weapon typeto determine whether the firearm has been raised to an aiming positionor out of the holster position.

In embodiments, a firearm usage monitoring system 2800 may allow a userto validate a threat, for example in a combat situation. A firearm usagemonitoring system 2800 may establish a pressure signature 3054 tovalidate the threat. The threat may be validated by the firearm usagemonitoring system 2800 by comparing the pressure signature against arange of pressure signatures, for example from no pressure to extremepressure.

In embodiments, the pressure signature 3054 may be established bycollecting information, from sensors, on or around the firearms and thelike. In embodiments, sensors may be wearable sensors 3058, such as froman armband, a watch, a wrist band, glasses, a helmet or other headgear,an earpiece, or the like, or may be combined with other sensors,including multi-modal sensors 3060. Sensors may also include otherwearable sensors, firearm motion sensors, firearm orientation sensors,firearm discharge sensors and combinations of sensors. Combinations ofsensors may include combinations of wearable and firearm sensors,combinations of firearms and fixed sensors, for example, Internet ofThings (IoT) sensors, and the like. A sensor-equipped firearm mayinclude a pressure sensor, for example, to determine a grip profileusing information such as threat ID, shot accuracy, engagement, alertinformation and tactical information. Information collected from asensor-equipped firearm may include discharge information, motioninformation, rate of motion information, orientation information and thelike. The rate of motion information may include movement informationrelated to speed, threat identification and shot accuracy. Movementinformation may also be related to an event identifier for events, suchas events associated with weapons and people. Events associated withfirearms may include events indicating the firearm has fallen, isoutside of a pre-designated distance from its owner, in an unauthorizedarea and the like. Events associated with people may include eventsindicating a person is in an unauthorized area, the maneuvering speed ofthe person and the like. Determining the pressure signature 1054 mayalso include determining a firearm-specific candidate action of a firstfirearm user, from at least a portion of the collected information. Thecandidate action may be compared with other firearm users, for example,other firearm users proximal to the first firearm user or other firearmusers associated with the first firearm user. The collected information,candidate action or actions, and action comparison result may then bestored in a data structure that represents the pressure signature 3054.The collected information, candidate action or actions, and actioncomparison result may also be filtered or weighted based on specifiedcriteria, prior to being stored in the data structure that representsthe pressure signature 3054.

In embodiments, the firearm usage monitoring system 2800 can providealternatives for monitoring discharges, such as cameras, or augmentsthose other monitoring systems. The methods and systems disclosed hereinmay include image recognition, which can identify the flash of a muzzleor for the slide rocking back. The system may also have acousticabilities and may provide sound recognition.

In embodiments, the firearm usage monitoring system 2800 can include aninfrared gate in front of the ejection port. This gate 3062 can track adisconnect when the weapon is fired, such as when the shell is engagedand breaks the gate 3062. In embodiments, the firearm usage monitoringsystem 2800 may include a hall effect sensor 3064 to measure the motionof an internal part. In embodiments, the firearm usage monitoring system2800 can capture the discharge profile of a given weapon by using anIMU. The discharge profile may have unique inertial characteristics whena weapon is discharged, such as based on the geometry, distribution ofweight, specified ammunition, and the like, so that a discharge can beprofiled and identified based on a series of movements that are measuredby the IMU. In embodiments, the firearm usage monitoring system 2800 maytrack with a global positioning system (GPS). In embodiments, thefirearm usage monitoring system 2800 includes network reportingfacility, such as through a Bluetooth discharge report to a centralizedserver. In embodiments, the firearm usage monitoring system 2800 canalso measure when a hand is on the grip of the weapon indicating athreatening situation. This sensor, button, or switch can providevaluable data, such as by alerting others to a potentially dangeroussituation.

In embodiments, the firearm usage monitoring system 2800 can include anactivity monitor which will indicate events such as when the gun iselevated and being pointed.

In embodiments, the firearm usage monitoring system 2800 can include aslim profile, waterproof enclosure to house the electronics and housing.In embodiments, the firearm usage monitoring system 2800 includes agrip-integrated reporting device including GPS technology. Inembodiments, the firearm usage monitoring system 2800 can be customizedwith various grip configurations and textures, such as to fit any kindof weapon with a familiar, comfortable type of grip that is typical forthat weapon.

In embodiments, the system 2800 can be integrated with other systems andaccessories. For example, a visible light (such as green or red) orinfrared laser pointing module 2814 can be integrated with the grip,such as to help with target acquisition, a flashlight to improvevisibility, or a range finder also for target acquisition.

In embodiments, the firearm usage monitoring system 2800 contains awireless charging system for the firearm discharge device to facilitategreater ease of use.

In embodiments, the firearm usage monitoring system 2800 can allow formanual or automatic calibration of the laser designator. In embodiments,the firearm usage monitoring system 800 can detect alternative trackingsystems when in a denied GPS location; for example, the system cantriangulate with cellular to provide an initial location to increase thespeed recognition of location or the system can triangulate with Wi-Fior other beacon technologies. In embodiments, the firearm usagemonitoring system 2800 can augment GPS with IMU to maintain relativeposition over time. The system can then provide better accuracy onphysical location within a building that cannot support GPS tracking. Inembodiments, the firearm usage monitoring system 2800 integrates withGPS-denied navigation systems.

In embodiments, the firearm usage monitoring system 2800 can augment thephysical location detection with depth sensors and camera systems togather data.

In embodiments, the firearm usage monitoring system 2800 can providedata storage. The system gathers data when the device is gripped throughminutes after the device is disengaged. If the device cannot transmit tothe edge device on the network (e.g., not available, out of range), itmay store (e.g., for up to 30 days) in onboard memory (e.g., throughhigh data rate memory). Once available, the system may restart thetransmission process, so that the data is sent over.

In embodiments, the firearm usage monitoring system 2800 has anecosystem for data. In embodiments, data may be aggregated, such as tocreate an aggregate database for firearms data, with various metricsthat can be applied to that kind of data, such as indicating groups orlocations that use weapons with varying frequency, that undertake moreor less training, and many others.

In embodiments, the firearm usage monitoring system 2800 can providepower management capabilities. If the device is in motion but not inuse, the low power mode (e.g., with occasional pinging) may beimplemented to maintain general awareness of the location of the user.The device transmits a location every one second. If not used for aperiod of time, (e.g., for ½ hour) the device may send one message at adefined interval, such as every second, every minute, every one-halfhour, every hour, or at other intervals.

In embodiments, the firearm usage monitoring system 2800 can provideinventory control. With monitoring, an alert can be sent and the weaponcan be tracked. Thus, for a manager, the system may provide locations ofall weapons of a given force at any given time.

In embodiments, the firearm usage monitoring system 2800 can providefirearm maintenance. With monitoring, the system may provide data on thenumber of rounds discharged and which gun components need maintenance orreplacement.

In embodiments, the firearm usage monitoring system 2800 can providereal-time tracking of users when in motion. This can identify where thedevice and users are at any time and when the weapon is in motion.

In embodiments, the firearm usage monitoring system 2800 can integratewith the body camera systems 2888 and automatically activates when thedevice is gripped or in motion. The body camera data can then bestreamed in real-time when in use. In embodiments, the firearm usagemonitoring system 2800 can be activated when motion is detected from thebody camera system 2888.

In embodiments, the firearm usage monitoring system 2800 can integratewith wearable devices 1058, such as activity monitors. It can integratewith mobile devices and the Emergency Response Data communicationsarchitecture.

In embodiments, the firearm usage monitoring system 2800 can includegeofence-based alerts. The geofence capability can be implemented arounda warehouse where weapons are stored to track weapons for inventorycontrol or threatening situations.

In embodiments, the firearm usage monitoring system 2800 can includepersonnel information including home addresses for location-basedreaction when granted permission where applicable.

In embodiments, the firearm usage monitoring system 2800 includes adashboard user interface 3068. Views available on the interface 3068 caninclude maps and can be populated with icons showing exact locations ofweapons. Each of the icons can include all personnel information for theweapon, status, and includes a button to zoom in on that location (anddrill down on the data). In embodiments, the firearm usage monitoringsystem 2800 provides aggregating units in the dashboard user interface3068. When the views on the inter-face 3068 become too dense withoverlapping icons, the map may adjust to include a new icon symbolizingmultiple units within the specific area.

In embodiments, the firearm usage monitoring system 2800 providessoftware-aided dispatch integration. The software used for monitoringfirearms can replace or augment the current computer-aided dispatchsystem to gain efficiency in call response and have one program to bemore effective.

In embodiments, the firearm usage monitoring system 2800 can integratewith Police Evidence Collection Systems, such as providing a centralizedsoftware suite that gathers the evidence information and can also allowcertain users to view and upload the information, creating efficienciesacross departments.

In embodiments, the firearm usage monitoring system 2800 can allowindividuals to review and replay firearm data as part of evidencecollection, training, and/or auditing purposes.

In embodiments, the firearm usage monitoring system 2800 can integratewith shooting ranges and retail point of sale (POS) inventory andmaintenance systems 3070.

In embodiments, the firearm usage monitoring system 2800 can integratewith the flight deck of an airplane and otherwise not be included in afirearm or system. The system 2800 may provide an IMU in the plane'ssteering wheel for further tracking purposes.

In embodiments, the firearm usage monitoring system 2800 can integratewith the controls of cargo ships, and the like. The system may providean IMU in the ship's steering wheel for further tracking purposes. Inembodiments, the system 2800 may be deployed to provide tracking withinshipping containers.

In embodiments, the firearm usage monitoring system 2800 can integratewith various vehicles and inventory to provide fleet and/or inventorymanagement.

In embodiments, the firearm usage monitoring system 2800 can adapt to alarge variety of firearms with various grip options.

In embodiments, the firearm usage monitoring system 2800 provides overthe air (OTA) updates for software upgrades.

In embodiments, the firearm usage monitoring system 2800 can integratewith original equipment manufacturer (OEM) components such as IMU, GPS,and Bluetooth.

In embodiments, the firearm usage monitoring system 2800 provide,integrate with, or connect to the machine control system 3000 andmachine-learning systems 3072 including custom algorithms fordetermining recoil of the firearm and other behaviors or characteristicsof the system. For example, in embodiments, the firearm usage monitoringsystem 2800 can include one or more machine learning systems 3072 withone or more identification methodologies to determine the complex motionassociated with the discharge of a particular type of weapon.Embodiments may include feeding IMU data collected upon gripping,movement, and discharge of weapons into the machine learning system3072, so that the system can learn the parameters of each with respectto enough training events that it can rapidly and accurately identifynew events based on new IMU data, such as collected in real time. Inembodiments, the system 3072 can be trained to learn to identify athreatening situation when the grip is engaged and the firearm ispointed, when the motion has increased indicating a pursuit, and when itis not in motion (e.g., placed in sleep mode). More complex patterns canbe learned, such as determining what patterns tend to lead to accidents,dangerous incidents, higher quality training, and the like.

In an example of learning and utilization of a complex pattern, afirearm usage monitoring system 2800 may use the machine learning system3072 to determine firearm movements that may indicate a discharge fromthe firearm is imminent. In this example, the machine learning system3072 may, for example, detect motion and orientation data from sensors,such as from sensors on the firearm 2020, sensors in the mesh network864 (including other firearms) or wearable sensors (e.g., multi-modalsensors) of the human user of the firearm, which in turn may be used bythe machine learning system 3072 to facilitate a threat response. Inembodiments, a threat response may include an automatic threat response,such as by one or more machines that are teamed with the human user ofthe firearm 2020.

In embodiments, the machine learning system 3072 may determinecombinations of data, such as motion, orientation and multi-modal sensordata that are indicative of imminent discharge of the firearm.

The machine learning system 3072 may also receive other inputs orgenerate information to combine with the sensor data, such as anindication of a firearm state. Firearm states may include combat states,training states, wartime states, peacetime states, civilian states,military states, first responder states, incident response states,emergency states, on-call states, and the like. Firearm states may bestates from one or more than one firearm, for example, a set of firearmsassociated with a group of soldiers in the same section of a battlefieldor a set of police officers in a region.

Combinations of data may allow the machine learning system to recognize,determine, classify, or predict information, such as about environments,objects, image content, whether a person is friendly or adversary,structures, landscapes, human and human gestures, facial indicators,voices, and locations, among others. Example combinations may includecombinations of data from topography and physiological monitors, ISR,and structure recognition combinations, as well as combinations of humanand machine physical states. Combinations of data may also be tacticalcombinations. Tactical combinations may combine data from devices on abattlefield, information about other sectors of fire, and the like andmay include firearms and other weapons, vehicles, body armor and otherwearable elements, and the like (collectively referred to herein as“battlefield of things”) devices including, for example, remotelyoperated units such as Common Remotely Operated Weapon Stations (CROWS)or other remote controlled firearms that may be configured with heaviercalibers and higher lethality.

Objects that may be recognized by machine learning may include weapons,man-made objects, natural objects, and the like. Structures may includedoors, stairs, walls, drop-offs, and the like. Human gestures may bedetected, interpreted and understood by the machine learning system,while facial indicators could be indicators of mood, intent, and thelike. The machine learning system 3072 may use thresholds to assist withdetermination and recognition process. For example, combinations of dataexceeding specified levels may provide a high degree of confidence thatthe recognition process is accurate.

In embodiments, the machine learning system 3072 teamed with the humanuser of the firearm 2020 may be operated autonomously, for example, inresponse to a determined intent of the human user of the firearm 2020teamed with the machine learning system 3072. The firearm usagemonitoring system 2800 may detect gestures of the human firearm user,such as by capturing and analyzing data from sensors that detectconditions of the human, as well as firearm sensors. Sensors that detectconditions of the human may include multi-modal sensors and multi-modalwearable sensors. Gestures may include pointing gestures, threatidentification gestures, target acquisition gestures, signaling gesturesand the like.

In embodiments, conditions recognized by the machine learning systems3072 or sensed in order to facilitate the training of the machinelearning system 3072 may include conditions indicative of human states,such as stress and other physiological states. Conditions indicative ofhuman states 3074 and captured by sensors for analysis by the firearmusage monitoring system may include heart rate conditions, for example,physical state relationships, blood pressure conditions, bodytemperature, galvanic skin response, heat flux, moisture, chemistry (forexample glucose levels), muscle states and neurological states. Variousbiological conditions or biosensors may be indicative of threats, suchas heart rate conditions, body temperature, moisture (such as indicatingexcessive perspiration), blood pressure, galvanic skin response, andothers. Firearm sensors may be multi-modal firearm sensors and mayinclude sensors that detect motion, orientation and discharge state ofthe firearm 2020.

Analyzing the data by the firearm usage monitoring system 2800 mayproduce a set of candidate intents 3080 of the human firearm user or ofanother individual in proximity to the firearm user (such as wherecamera information, voice information, and the like is available). Thecandidate intents 3080 may, in embodiments, be combined with physicaland operation machine state information to select one or more actionplans 3082. The machine teamed with the human user of the firearm 2020may then execute and adjust the selected action plan 3082 based onupdated intents, machine states, and environmental factors. Machinestate factors may include physical factors, operational factors,orientation factors, tactile/force factors, and the like.

Environmental factors 3084 may include weather factors, location datafactors, altitude factors, topography factors, video factors and thelike. Weather factors may include temperature, humidity, wind speed,wind direction and precipitation factors, among others. Location datafactors may include streaming data, as well as data acquired from globalpositioning systems (GPS) and beacons, access points or the like, aswell as through cellular triangulation. Topography factors may includedata and observations, while video factors may include both live andarchived video feeds. The action plan 3082 may also be formed from a setof predetermined action steps, for example, action steps that eachsatisfy human teaming criteria selected to coordinate with at least oneof the candidate intents 3080. Actions steps may also be arranged intoaction plans by sets of rules.

With reference to FIG. 31 , the machine learning system 3072 may includethe machine control system 3000 that may team with a human user of afirearm. In embodiments, the machine control system 3000 may receivemulti-modal sensory input 3002 from multi-modal sensors. The multi-modalsensory input 3002 may send sensed data to a sensory analysis module1004. The sensory analysis module 3004 may forward an actionablerepresentation of the sensed data to a control scheduling process module3006 and a real-time control process module 3008 for further processing.

The control scheduling process module 3006 may provide schedulingcontrol information to the real-time control process module 3008 thatmay issue machine control scenarios to machine controller modules 3010.The machine control modules 3010 may affect the machine controlscenarios, for example, by mechanization of the machine through a finalcontrol element module 3012. Machine control scenarios may includerecognition of celebratory situations such as dancing scenarios andfirst bump scenarios separate from other human machine learningscenarios in much more threatening and complex environments. In manyexamples, the machine learning system 3072 may identify celebratory fireover threatening fire. In embodiments, one or moreanalysis-schedule-real-time modules 3088 (FIG. 33 ) may storeinformation in a storage module 3014 for use as feedback/input to themachine learning system, such as feedback provided through feedbackmodules 3016, that then may adjust parameters for teaming. It will beappreciated in light of the disclosure that it may not be practical tohard code every combination of movement and therefore the machinelearning system 3072 may be configured to identify one or more series ofmovements after being shown by one or more human users of other machinelearning systems. By way of these examples, the machine learning system3072 may learn the movements of the its users by translating anddetecting their motion and comparing the identified motions in contextwith the environment in comparison with trained examples, confidence inthose examples, corrections to past activity, and the like to assist,anticipate, protect, support, and facilitate the needs of the users inthe theater more quickly and more safely.

In many examples, social interactions between human users and machinesdeployed with them must be learned by both parties. It will beappreciated that early stage robots (i.e., those incapable of expressing“feelings”) could improve the psyche of their human counterpart evenwith little mutual social interaction. With that said, many situationsarise where mutually beneficial social interactions between the usersand the machine learning system 3072 may improve the ability of themachine learning system 3072 to assist, anticipate, protect, support,and facilitate the needs of the users in the theater more quickly andmore safely. Many situations are additionally good candidates to trainthe machine learning system 3072 to understand friendly environmentsover threatening situations. In these environments and situations, themachine learning system 3072 may need to learn how to interact more withhuman users in order to better produce a more intuitive experience. Inmuch the same way as our homes may be associated with a certain smell orfeeling, the machine learning system 3072 may need to understand andrelate sensory inputs with other inputs and schedule specific actionsand processes. If a human user and robotic machine counterpart enter themess hall which is not a combat zone, the machine learning system 3072would need to understand that a different set of actions or schedulingprocesses occurs in this environment when instructing its roboticmachine counterparts (or other assets) in the area.

In embodiments, the machine learning system 3072 may manage acoordinated team of human users of firearms and at least one machine. Inthis embodiment, the machine learning system 3072 may receive as inputsat least one sensory input about a human and at least one sensory inputabout a machine that is part of the team coordinated with the human. Themachine learning system 3072 may then automatically, using machinelearning, determine the occurrence of an event, such as a pre-dischargeevent, a discharge event, a post-discharge event (including apost-discharge adverse event) or other events. The post-dischargeadverse events may include injury to the human or occurrence of damageto the machine, such as subsequent to the detection of a firearmdischarge event by the system.

In embodiments, the firearm usage monitoring system 2800 may be orinclude an all-in-one communication device 3090. The system mayintegrate with a variety of other communication devices, such as camerasystems 2888 including body cameras, helmet cameras, heart ratemonitors, physiological monitors, and messaging.

In embodiments, the firearm usage monitoring system 2800 may integratewith physiological monitors. A heart rate band or monitor can be anindicator of a distressed situation creating a notification.

In embodiments, the firearm usage monitoring system 2800 can integratewith mobile phone technology. The system can send critical messages in atimely manner, such as through an app that may be directly connected todispatchers, such as allowing the caller to request assistance.

In embodiments, the firearm usage monitoring system 2800 may provide adashboard for the dispatcher. The dashboard may include communicationand mapping features, such as to track the location of all weapons inreal-time, to highlight relevant events (such as weapons being gripped,weapons being raised, or weapons that have been discharged). Thedashboard may provide access information from other systems, such asmaking available camera views, such as ones that are triggered byactivation of body cameras or on-site cameras from the firearmmonitoring system or from the dashboard. In embodiments, the firearmusage monitoring system 2800 provides a dashboard for the supervisor. Inembodiments, the dashboard includes the communication system and mappingtechnology to track the location of all weapons in real-time. Inembodiments, the firearm usage monitoring system 2800 separates usersinto groups/echelons with designated permissions. In embodiments, thefirearm usage monitoring system 2800 provides a dashboard for one ormore of ground units, officers, military personnel, aninvestigator/compliance officer, and the like. The dashboard may includethe communication system and mapping technology to track the location ofall weapons in real-time.

In embodiments, the firearm usage monitoring system 2800 measures theparameters of the recoil and parameters of pre-shot movement. Thisallows an analysis of changes over time to determine the status of theweapon. The system can also capture movements and determine whether theuser is handling the weapon properly.

In embodiments, the firearm usage monitoring system 2800 may alert theuser should the weapon be pointed at another person with a trackingsystem. The firearm usage monitoring system 800 may also alert the usershould the weapon be pointed at another weapon, another deployed asset,another predefined target, raised quickly in a geo-defined zone, or thelike. This may help avoid friendly fire (fratricide) situations.

In embodiments, the firearm usage monitoring system 2800 integrates witha virtual, augmented, or heads-up display (HUD) reality system 3092including virtual, augmented reality, or HUD glasses. This integrationcan provide the user with vital information, including how many roundsof ammunition are left, such as based on tracking discharges over timeand comparing to known characteristics of a weapon, such as the size ofa magazine.

In embodiments, the firearm usage monitoring system 2800 includespredictive maintenance, such as determined by the number of shots taken.The system can alert when components need to be maintained or replaced.

In embodiments, the firearm usage monitoring system 2800 allows thenumber of shots fired to influence the resale value of the firearm.

In embodiments, the firearm usage monitoring system 2800 includespredictive maintenance based on recoil parameters (e.g., showing adegradation of performance as recoil patterns shift over time).

In embodiments, the firearm usage monitoring system 2800 includes apredictive resupply module 3094 based on the number of shots taken. Inembodiments, the firearm usage monitoring system 2800 indicates whenammunition needs to be re-supplied.

In embodiments, the firearm usage monitoring system 2800 accounts for aninventory of rounds used with the predictive resupply module 3094 thattracks the amount of ammunition used and alerts when the inventory andshots fired do not match indicating a loss of ammunition.

Methods and systems are provided for the installation of grips. Thefireguards can be removed to install the tracking system on to therails. Firearm grips have many ornamental features separate and distinctfrom their many functional features.

In embodiments, the firearm usage tracking system integrates an IMU intoa smart weapon (e.g., one with user authentication, such as based on apassword or other code, or a biometric authentication system).

In embodiments, the firearm usage monitoring system 2800 can include agrip-located IMU for a connected firearms platform.

In embodiments, the firearm usage monitoring system 2800 can integratewith artificial intelligence (AI) and Machine Learning. For example, AIcan provide predictive ammunition re-supply, such as measuring firerates and accounting for the delivery time of new ammunition.

In embodiments, the firearm usage monitoring system 2800 can integratewith virtual reality (VR) or augmented reality (AR) using, for example,a Microsoft® HoloLens® for training purposes. A virtual command centerfor a battlefield training session can be created.

In embodiments, the firearm usage monitoring system 2800 can provide VRand AR grip installation. VR video can be used to identify the platformand provide instruction on the removal and installation of grips and orother firearm parts.

In embodiments, the firearm usage monitoring system 2800 can supply datato an AR/VR system 1098 that included VR and AR headsets. This may allowusers to monitor inventory, rounds left in the magazine, and otherrelevant data including a map of the environment and surrounding unitsand objective markers.

In embodiments, the firearm usage monitoring system 2800 can havecustomizable grips provided through 3D printing or other manufacturingprocesses. Each individual can customize a style, color, texture,portions of shapes, concavity and convexity to better fit in the hand,changing knurled surfaces, combinations of textures and colors andpurposely different designs and configurations, etc. on one side thegrip relative to the other or make them mirror images of each other.

In embodiments, the methods and systems disclosed herein providebenefits to a wide number of users, including without limitation privateand commercial gun users. One such set of users comprises of managers offirst responder and law enforcement personnel, such as police chiefs andelected officials that manage officers and dispatchers.

A firearm which implements or otherwise integrates one or more of themethods and systems disclosed herein (e.g., one or more of the firearmusage monitoring system 2800, the firearm tracking system 2882, themachine learning system 3072, or another system) includes one or morestructures for performing or facilitating operations typical of afirearm, for example, for storing ammunition, firing one or moreprojectiles from the ammunition, controlling the storage and firing ofammunition, and more. In embodiments, a firearm which implements orotherwise integrates one or more of the methods and systems disclosedherein can include an action structure, a stock structure, and a barrelstructure. In embodiments, a firearm which implements or otherwiseintegrates one or more of the methods and systems disclosed herein caninclude one or more rails. A rail may, for example, be located on one ormore of, or proximate to one or more of, the action structure, the stockstructure, or the barrel structure.

FIGS. 20, 21 and 22 show a first example of a firearm 2020 whichimplements or otherwise integrates one or more of the methods andsystems disclosed herein (e.g., one or more of the firearm usagemonitoring system 2800, the firearm tracking system 2882, the machinelearning system 3072, or another system). However, the methods andsystems disclosed herein may be implemented or otherwise integratedwithin other types of firearms or other firearm form factors.

In embodiments, a firearm which implements or otherwise integrates oneor more of the systems disclosed herein (e.g., the firearm 2020, thefirearm 3100, or another firearm) can include structures other than anaction structure, a stock structure, a barrel structure, and/or one ormore rails. For example, in embodiments, such a firearm can include acylinder structure including multiple chambers for storing a projectileto be fired. For example, the firearm may be a revolver or anotherfirearm with a structure for rotating multiple chambers into alignmentwith the bore of the barrel structure. In another example, inembodiments, such a firearm may omit the stock structure. For example,the firearm may be a pistol or other handgun in which components such asthe grip and/or trigger are coupled to the rest of the firearm by astructure other than a stock structure. In another example, inembodiments, such a firearm may include a stock structure that omits thebutt. For example, the firearm may be a pistol or other handgun whichincludes a stock structure that structurally supports the actionstructure and/or the barrel structure, but in which contact with theuser is intended to be limited to the grip. It is to be understood thatother firearm embodiments as are currently known or which are laterdeveloped may be used to implement or otherwise integrate one or more ofthe methods and systems disclosed herein.

In embodiments, the firearm 104 is a smart weapon configured to preventdischarge if predetermined criteria are not met. For example, thefirearm 104 may be associated with a personal device, such as a mobiledevice, such that the firearm 104 will not discharge unless the mobiledevice is within a predetermined distance of the firearm. Beneficially,such coupling is also beneficial because the mobile device may be sentan alert if the firearm 104 is moved or otherwise altered while themobile device is more than a predetermined distance away. Beneficially,the firearm usage monitoring system or a connected system may be used tosend a temporary deactivation message to the firearm 104 (e.g., lawenforcement deactivating the firearm of an active shooter) to preventcriminal usage of the firearm. Further, the firearm 104 may be geofencedout of areas where legal firearm 104 usage is unlikely or poses a dangerto the user or others (e.g., public areas, crowded events, etc.).

The firearm may further include indicators on the weapon, such as aplurality of LEDs on the grip, that provide information to a user of thefirearm 104, such as an operating condition, malfunction condition,maintenance condition, etc.

Components used by one or more of the methods and systems disclosedherein may be located or otherwise positioned with respect to certainstructures and/or certain components of structures of a firearm (e.g.,the firearm 2020, the firearm 3100, or another firearm). For example, anIMU used by one or more of the methods and systems disclosed herein maybe coupled at one or more locations or positions of a firearm. Inembodiments, the IMU may be coupled to or included in the charginghandle. In embodiments, the IMU may be coupled to the forward assistcomponent. In embodiments, the IMU may be coupled to the gas operatingsystem. In embodiments, the IMU may be coupled to or included in thehammer. In embodiments, the IMU may be coupled to or included in aportion of the action structure other than as described above. Inembodiments, the IMU may be coupled to or included in the butt. Inembodiments, the IMU may be coupled to or included in the grip of thebutt. In embodiments, the IMU may be coupled to or included in the combof the butt. In embodiments, the IMU may be coupled to or included inthe hook coupled to the butt. In embodiments, the IMU may be coupled toor included in the fore-end. In embodiments, the IMU may be coupled toor included in a handguard of the fore-end. In embodiments, the IMU maybe coupled to or included in the trigger unit. In embodiments, the IMUmay be coupled to or included in the magazine well. In embodiments, theIMU may be coupled to or included in the magazine received in themagazine well. In embodiments, the IMU may be coupled to or included ina portion of the stock structure other than as described above. Inembodiments, the IMU may be coupled to the external surface of thechamber. In embodiments, the IMU may be coupled to the chamber at alocation or position other than the external surface. In embodiments,the IMU may be coupled to an exterior surface of the bore. Inembodiments, the IMU may be coupled to the bore at a location orposition other than the external surface. In embodiments, the IMU may becoupled to an external surface of the muzzle. In embodiments, the IMUmay be coupled to or included in an accessory device coupled to themuzzle, for example, in which the muzzle includes a coupling element(e.g., a threaded or other engagement) for coupling the accessory deviceto the muzzle. In embodiments, the accessory device may be coupled to aportion of an external surface of the barrel structure other than anexternal surface of the muzzle. In embodiments, the IMU may be coupledto or included in the muzzle at a location or position other than theexternal surface or the coupling element to which an accessory devicemay be coupled. In embodiments, the IMU may be coupled to or included ina portion of the barrel structure other than as described above. Inembodiments, the IMU may be coupled to or included in a scope coupled toa rail of the firearm. In embodiments, the IMU may be coupled to orincluded in a sight coupled to the rail. In embodiments, the IMU may becoupled to or included in a tactical light coupled to the rail. Inembodiments, the IMU may be coupled to or included in a vertical forwardgrip coupled to the rail. In embodiments, the IMU may be coupled to orincluded in a portion of a rail or an accessory coupled to a rail otherthan as described above. It is to be understood that examplesparticularly referring to the IMU do not limit the possible embodimentsof other components used by one or more of the methods and systemsdisclosed herein being coupled at one or more locations or positions ofa firearm.

In embodiments, components used by one or more of the methods andsystems disclosed herein which may be located within or otherwisepositioned with respect to the structures described above may, forexample, include an IMU. In embodiments, the IMU may be coupled to orincluded in outerwear. In embodiments, the IMU may be coupled to orincluded in a helmet. In embodiments, the IMU may be coupled to orincluded in an earpiece. In embodiments, the IMU may be coupled to orincluded in glasses. In embodiments, the IMU may be coupled to orincluded in one or more wristbands or wristwatches. In embodiments, theIMU may be coupled to or included in other wearable items. Whileexamples of particular structures of a firearm and particular componentsof structures of a firearm are disclosed herein, such disclosure is notlimiting as to the possible structures of components of structures of afirearm or as to the possible locations or positionings of componentsused by the methods and systems disclosed herein with respect to thosestructures or those components of structures. Accordingly, it is to beunderstood that components used by one or more of the methods andsystems disclosed herein may be located or positioned in other locationsor positions in or about a firearm, regardless of the particularstructures disclosed herein by example.

With additional reference now to FIGS. 42A-49 , multiple event detectionmodules configured for use with various types of firearms will bedescribed. The event detection modules are further configured formounting at different locations on a firearm such as on a grip, a railand a barrel of a firearm as will be described. In additional examples,as mentioned above, an event detection module can be included on awearable device, such as a wristband, wristwatch, or other article wornby the user. As shown in FIGS. 42A-44 an event detection module 4200A isprovided within a grip housing 4202A. The event detection module 4200Aand the grip housing 4202A are collectively referred to herein as a gripmodule 4204A. The grip housing 4202A can include an outer contoured body4206A (FIGS. 43 and 44 ) configured for receiving a trigger hand of auser of the firearm. The event detection module 4200A is generallydisposed in an event detection module housing 4207A. It will beappreciated that the event detection module 4200A and any of the otherevent detection modules disclosed herein can be provided alone or as aunit with an event detection module housing such as the event detectionmodule housing 4207A. In examples, the event detection module 4200A canbe provided as a standalone unit (with or without an event detectionmodule housing 4207A) for mating with a particular firearm. Asdiscussed, these standalone units can be adapted for receipt by areceiver of a firearm such as at a grip (e.g., grip housing), a rail, orother locations of a firearm (e.g., barrel housing). The event detectionmodules discussed herein can make determinations of a type of event(e.g., shot occurring) based on various sensor inputs (inputs fromsensors 118 that can include an IMU and/or other sensors as describedherein). The determinations can be made in real time and saved oninternal circuitry and/or be communicated wirelessly to the server 112(FIG. 1 ) such as for additional data analysis.

With reference to FIG. 44 , the event detection module housing 4207A canbe additionally mechanically coupled to the grip housing 4202A by a pinor fastener 4208A that extends through respective apertures 4208B and4208C in the event detection module housing 4208A and the grip housing4202A. Additional or alternative mechanical coupling configurations maybe included on the event detection module housing 4207A such as a quickconnect, magnetic coupling, screw coupling and press-fit coupling. Abattery 4209A that provides power to the event detection module 4200E isgenerally housed in the event detection module housing 4207E. Inexamples, the event detection module 4200A can be easily removed andreplaced with another event detection module 4200A from the grip housing4202A.

In examples, the grip module 4204A is configured to be selectivelycoupled to various long guns 4210A-4210D (FIGS. 42A-42D). By way ofexample only, the long gun 4210A (FIG. 42A) can be an M5. The long gun4210B (FIG. 42B) can be an M250. The long gun 4210C (FIG. 42C) can be alight machine gun (LMG). The long gun 4210D (FIG. 42D) can be a mediummachine gun (MMG). It will be appreciated that the long guns 4210A-4210Dare merely exemplary and the grip module 4204A may be configured for usein other firearms.

As will be described herein, the event detection module 4200A cancomprise the sensors 118 (FIG. 1 ) such as one or more IMU's eachincluding an accelerometer and a gyroscope, a magnetometer, and ageolocation sensor. In examples, two IMU's having distinct sensitivitiescan be included that provide acceleration and/or rotational inputs tothe event detection module 4200A. Additional sensors of the sensors 118can include a pressure sensor, an audio sensor, a thermal sensor,combinations thereof, and the like. The sensors 118 are generallyprovided on a circuit board 4211A. The event detection module 4200A caninclude a signal processing module (4430, FIG. 54C) that uses machinelearning and/or digital signal processing techniques to determine aclassification of an event detected such as a discharge event of thefirearm. In additional examples, the event detection module 4200A canwirelessly communicate signals ultimately to the server device 112 forprocessing to make additional determinations based on the event. Inexamples, the server device 112 can collect signals from a plurality ofevent detection modules in the field from many weapons to make tacticaland/or training decisions.

Referring now to FIGS. 43 and 44 , the event detection module 4200Aincludes a USB interface 4214A, a user-operated switch 4216A and a lightemitting member 4218A all generally provided on a user interface panel4220A. The user-operated switch 4216A can be moved between activated(“ON”) positions and deactivated (“OFF”) positions corresponding to a“use” position where signals are communicated from the event detectionmodule 4200A and a “go dark” position where no signals (such as radiofrequency signals) are communicated from the event detection module4200A, resulting in zero detection. A cover 4222A (FIG. 44 ) can berotatably coupled to the grip housing 4202A for moving between open andclosed positions. The grip module 4204A is configured to be coupled to arespective receiver 4224A-4224B of each of the firearms 4210A-4210D. Inuse, a user may move the user-operated switch 4216A to the “OFF”position to preclude any signal emitting from the event detection module4200A. Notably, however, with the user-operated switch in the “OFF”position, the event detection module 4200A still receives data from thesensors 118 and provides signal processing steps as described herein. Inother words, the event detection module 4200A (and other event detectionmodules described herein) operates as normal to provide and locallystore event detection information (shot detection, etc.), but do notsend wireless transmissions of data and/or detection information such asto a network 114 (FIG. 1 ). As can be appreciated, a user, such as asoldier wishing to reduce their electromagnetic (EM) signature can movethe switch 4216A to “OFF” while still conducting data acquisition andstoring locally at the event detection module. In examples, when a useris in a situation where reduction of EM signature is no longer required,the switch 4216A may be moved back to the “ON” position and the eventdetection module will offload (e.g., send to the network 114) any dataand/or event detection information collected while in the “OFF”position. It is appreciated that while the exemplary switch 4216A isshown as a rocker/toggle switch, it may be configured differently (e.g.,a button, etc.). In other examples, radio transmission can be disabledand enabled differently. In some examples, radio frequency signalsdefault to OFF but can be communicated post mission during an offloadingof data event. In one example, radio frequency signals are communicatedonly upon connection to a power source such as a battery and/or to a USBinterface 4214B post mission, such as during a charging event. Inanother example, the switch 4216A can be replaced by a firmwaretechnique executed by processors in the event detection module 4200.With particular reference now to FIGS. 42C, 42D and 45 , another eventdetection module 4200B is provided. The event detection module 4200B isconfigured to be coupled to a quick change barrel assembly 4228. Theevent detection module 4200B is provided within a grip housing 4202B.The event detection module 4200B and the grip housing 4202B arecollectively referred to herein as a grip module 4204B. The grip housing4202B can include an outer contoured body 4206B (FIG. 45 ) configuredfor receiving a hand of a user of the firearm. The grip housing 4202B isconfigured as a handle for generally carrying or maneuvering the firearmas a whole. The event detection module 4200B is generally disposed in anevent detection module housing 4207B. The grip module 4204B can begenerally mounted on an arm 4230 extending from a barrel 4232 of thequick change barrel assembly 4228.

In examples, the grip module 4204B is configured to be selectivelycoupled to the long guns 4210C and 4210D (FIGS. 42C and 42D). As withthe event detection module 4200A, the event detection module 4200B cancomprise the sensors 118. The sensors are provided on a circuit board4211B. The event detection module 4200B can include a signal processingmodule that uses machine learning and/or digital signal processingtechniques to determine a classification of the detected event such aswhether an event detected is a discharge event of the firearm. In otherexamples, the event detection module 4200B can wirelessly communicatesignals ultimately to the server device 112 for processing to makeadditional determinations based on the event.

The grip module 4204B is particularly advantageous in that shots can becounted specific to the barrel 4232. In this regard, maintenanceinformation can be determined specific to the barrel 4232 (as opposed tothe firearm as a whole). In examples, shot counts can be tallied by theevent detection module 4200B and/or by the server device 112 and adetermination can be made to replace the barrel 4232 for a new barrel4232 such as when a shot count threshold has been reached. In examples,a barrel may have a reduced lifespan relative to the firearm as a whole.In this regard, a preventative maintenance item associated with such afirearm can include replacing a barrel in a quick change barrel assembly4428 that has reached an expected shot count lifespan. Otherinformation, discussed herein, can be determined specific to the healthof the barrel 4232 based on signals communicated from the sensors 118 inthe event detection module 4200B. Such additional information caninclude thermal readings detected by sensors 118 and communicated to theserver device 112. In such examples, the sensors 118 can include athermal sensor. The event detection module 4200B can generate an eventdetection signal based on the temperature sensed at the thermal sensorindicative of a temperature of the barrel 4232. Additionally oralternatively, calculations can be made by the event detection module4200B at the server device 112 that estimate a temperature of the barrel4232 based on environmental conditions including ambient temperature atthe thermal sensor, the shot count, and the rate of shot discharge toestimate a temperature of the barrel 4232. In other examples, a numberof shots can be determined in addition to the rate of shot dischargesassociated with the barrel 4232. In examples, each barrel can beassociated with a specific barrel identification number. The eventdetection module 4200B can determine (and store and/or wirelesslycommunicate) shot counts and additional information (thermal readings)specific to a particular barrel of the firearm. In this regard, a commonevent detection module 4200B can collect and store information formultiple distinct barrels.

The event detection module 4200B includes a USB interface 4214B, auser-operated switch 4216B and a light emitting member 4218B (such as alight emitting diode) all generally provided on a user interface panel4220B. The user-operated switch 4216B can be moved between activated(“ON”) positions and deactivated (“OFF”) positions corresponding to ause position where signals are communicated from the event detectionmodule 4200B and a “go dark” position where no signals are communicatedfrom the event detection module 4200B. A cover 4222B can be removablyattached, such as threadably, to the grip housing 4202B. While the eventdetection module 4200B is described herein with respect to the quickchange barrel assembly 4228, it is appreciated that the event detectionmodule 4200B can be configured for use with other barrels of otherfirearms within the scope of this disclosure.

With reference now to FIGS. 46A and 46B, another event detection module4200C is provided. The event detection module 4200C is configured to bedisposed within a grip 4248 of a hand gun body 4250. In the exampleshown, the hand gun body 4250 is collectively defined by clam shell bodyportions 4250A and 4250B. As with the other event detection modulesdisclosed herein, the event detection module 4200C can comprise thesensors 118 (FIG. 1 ) such as an IMU including an accelerometer, agyroscope, a magnetometer, a geolocation sensor, a pressure sensor, anaudio sensor, combinations thereof, and the like. The sensors 118 aregenerally provided on a circuit board 4211C. The event detection module4200C can include a signal processing module that uses machine learningand/or digital signal processing techniques to determine aclassification of an event such as whether an event detected is adischarge event of the firearm. While not specifically identified, theevent detection module 4200C can include additional features such asdescribed above with respect to the event detection module 4200A such asa USB interface and a battery for powering the event detection module4200C. In examples, the event detection module 4200C can wirelesslycommunicate signals ultimately to the server device 112 for processingto make additional determinations based on the event as described above.

With particular reference now to FIG. 47 , another event detectionmodule 4200D is provided. The event detection module 4200D is providedwithin a grip housing 4202D. The grip housing 4202D includes a coverportion 4208D that is fixedly disposed on a main grip body 4210D. An endcap housing 4212D can be engaged to the main grip body 4210D. Inexamples fasteners can couple the grip housing 4202D and the end caphousing 4212D together. An O-ring (not shown) can additionally locatebetween the grip housing 4202D and the end cap housing 4212D. The eventdetection module 4200D and the grip housing 4202D are collectivelyreferred to herein as a grip module 4204D. The grip module 4204D can beconfigured for use with rifles such as, but not limited to an M41 pulserifle. The grip housing 4202D can include an outer contoured body 4206Dconfigured for receiving a hand of a user of the firearm. The eventdetection module 4200D is generally disposed within the grip housing4202D. A user-operated switch 4216D such as a toggle switch or buttoncan be provided on a user interface panel 4220D. The user-operatedswitch 4216D can be moved between activated (“ON”) positions anddeactivated (“OFF”) positions corresponding to a use position wheresignals are communicated from the event detection module 4200D and a “godark” position where no signals are communicated from the eventdetection module 4200D.

As with the other event detection modules disclosed herein, the eventdetection module 4200D can comprise the sensors 118 (FIG. 1 ) such as anIMU including an accelerometer, a gyroscope, a magnetometer, ageolocation sensor, a pressure sensor, an audio sensor, combinationsthereof, and the like. The sensors 118 are generally provided on acircuit board 4211D. The event detection module 4200D can include asignal processing module that uses machine learning and/or digitalsignal processing techniques to determine a classification on an eventsuch as whether an event detected is a discharge event of the firearm.In examples, the event detection module 4200D can wirelessly communicatesignals ultimately to the server device 112 for processing to makeadditional determinations based on the event.

With particular reference now to FIGS. 48A-49 , another event detectionmodule 4200E is provided. The event detection module 4200E is providedwithin a grip housing 4202E. The event detection module 4200E and thegrip housing 4202E are collectively referred to herein as a grip module4204E. The grip module 4204E is configured on a spade grip assembly4260. While the spade grip assembly 4260 generally comprises two griphandles, it is contemplated that the grip module 4202E can be configuredon one of the two grip handles. The spade grip assembly 4260 is shown onan exemplary firearm 4210E such as an MK19 however it is appreciatedthat the grip module 42024E can be used in other firearms. The spadegrip assembly 4260 can be configured on a back plate assembly 4261 thatcouples to a proximal end of the firearm 4120E.

With particular reference to FIG. 49 , the grip housing 4202E generallyincludes first and second clam shell housing portions 4262A and 4262B,and end screws 4264A and 4264B. The event detection module 4200E isgenerally disposed in an event detection module housing 4207E. Inexamples, the event detection module housing 4207E can define threadedbores 4208A and 4208B for receiving the end screws 4264A and 4264B,respectively. A battery 4209 that provides power to the event detectionmodule 4200E is generally housed in the housing 4207E.

As with the other event detection modules disclosed herein, the eventdetection module 4200E can comprise the sensors 118 (FIG. 1 ) such as anIMU including an accelerometer, a gyroscope, a magnetometer, ageolocation sensor, a pressure sensor, an audio sensor, combinationsthereof, and the like. The sensors 118 are generally provided on acircuit board 4211E. The event detection module 4200E can include asignal processing module that uses machine learning and/or digitalsignal processing techniques to determine a classification of the eventsuch as whether an event detected is a discharge event of the firearm.In other examples, the event detection module 4200E can wirelesslycommunicate signals ultimately to the server device 112 for processingto make additional determinations based on the event as described above.

With particular reference now to FIGS. 50-53B, another event detectionmodule 4200F is provided. The event detection module 4200F is providedwithin a grip housing 4202F. The event detection module 4200F and thegrip housing 4202F are collectively referred to herein as a grip module4204F. As will become appreciated, the grip module 4204F can beconfigured similarly to the grip module 4204A described above and shownat FIG. 44 , but is also configured for cooperation with a trigger pullsensor assembly 4300. As shown in FIG. 2 , an event detection module4200G is integrated within a wearable device, such as a wristband (seewristband 226), a wristwatch, or other article worn by the user.

The trigger pull sensor assembly 4300 can provide an additional sensorinput to the event detection module 4200 when detecting an event such asshot count. In this regard, information provided by the trigger pullsensor assembly 4300 can be combined with signals from other sensors(e.g., IMU) discussed herein to provide yet another layer of input toachieve a more robust determination of the existence of an event such asa discharge event. In other examples, while the trigger pull sensorassembly 4300 will provide a signal indicating that the trigger has beenpulled, it may not necessarily indicate that the firearm has discharged.Explained further, if it is determined that the trigger pull sensorassembly 4300 has provided a first signal that satisfies a thresholdindicative of the trigger being pulled in combination with the sensors118 (e.g., IMU) providing a second signal indicative of a dischargeevent, a determination may be made that a discharge of the firearm hasoccurred. Conversely, if it is determined that the trigger pull sensorassembly 4300 has provided a first signal indicative of the triggerbeing pulled but the sensors 118 do not provide a signal indicative of adischarge event, a determination may be made that the firearm has one ofmalfunctioned, an ammunition status such as if the weapon is out ofammunition, or another failure is present. In examples, thesedeterminations can be communicated to a user of the firearm inreal-time, such as on a display (for example the AR/VR display 5400described herein and show in FIG. 70 ). In yet further examples, asignature of the signal generated from the sensors 118 can be analyzedto correlate the event to a particular type of malfunction. Inadvantages, the incorporation of the trigger pull sensor assembly 4300as an additional input to the event detection module 4200F improvesaccuracy of a discharge/shot count determination and reduces falsepositives. In examples, analyzing the second signal provided by thesensors 118 is only conducted based on the trigger pull sensor assembly4300 initially providing the first signal indicative of the triggerbeing pulled. It is further contemplated that such discharge/shot countinformation can be provided in real time allowing the rapid use ofcollected information, such as for situational awareness and rapidresponse to critical situations. The information can be output to any ofthe GUI described herein (FIGS. 6-11 ) to generate, review and/orapprove actions to take in response to the event detection.

The trigger pull sensor assembly 4300 generally comprises a grip screw4308 that generally houses a plunger assembly 4310 (FIGS. 53A-53B). Theplunger assembly 4310 mechanically communicates motion of a trigger 4314to motion of a contact sensor 4320. The grip screw 4308 threadablyconnects to a body of a firearm 4210F. In examples, the grip screw 4308defines a pocket 4322 (FIG. 53B) that accommodates a portion of thecontact sensor 4320. As will be described, the trigger pull sensorassembly 4300 senses mechanical movement of the trigger 4314 andcommunicates a trigger actuation signal to the event detection module4200F corresponding to a position of the trigger 4314. The eventdetection signal is further based on the trigger actuation signal.

The contact sensor 4320 is configured to move (e.g., translate) on thegrip housing 4202F between a depressed position (FIG. 53A) and anextended position (FIG. 53B). The contact sensor 4320 is normally biasedoutward by a contact sensor biasing member 4330 housed in the griphousing 4202F. When the grip housing 4202F is installed into the gripmodule 4202F, the depressed position (FIG. 53A) corresponds to thetrigger 4314 in a ready (non-pulled) position. The extended position(FIG. 53B) corresponds to the trigger 4314 in an actuated (pulled)position. In the example provided, movement of the contact sensor 4320from the depressed position (FIG. 53A) to the extended position (FIG.54B) causes a circuit on the event detection module 4200F to openindicative of a shot being fired. It is appreciated that the circuit canbe configured differently whereby the circuit can close indicative of ashot being fired. In other words, the contact sensor 4320 can changestates on the circuit indicative of the trigger 4314 being pulled.

Turning now to FIG. 52 , an exemplary trigger assembly 4370 is shown.The trigger assembly 4370 generally comprises the trigger 4314, adisconnector 4372, and a hammer 4373. The hammer 4373 can define a sear4378 for mechanically communicating with a hook 4380 on the disconnector4372. With additional reference to FIGS. 53A and 53B, the plungerassembly 4310 will be further described. The plunger assembly 4310includes a trigger side plunger 4390, a sensor side plunger 4392 and aplunger biasing member 4394. The trigger side plunger 4390 is configuredto engage the trigger 4314 while the sensor side plunger 4392 isconfigured to engage the contact sensor 4320. When the trigger 4314 isin the ready (non-pulled) position, the plunger biasing member 4394provides sufficient biasing force between the trigger side plunger 4390and the sensor side plunger 4392 to urge the sensor side plunger 4392toward the contact sensor 4320 (and overcome the bias from the contactsensor biasing member 4330, FIG. 51B) to maintain the contact sensor4320 in the depressed position (FIG. 53A). When the trigger 4314 ispulled (FIG. 53B), effectively rotating around a pivot 4396, the plungerassembly 4310 collectively translates away from the contact sensor 4320such that the contact sensor 4320 moves from the depressed position(FIG. 53A) to the extended position (FIG. 53B). In other words, the biasof the contact sensor biasing member 4330 overcomes the bias of theplunger biasing member 4394 to move the contact sensor 4320 to theextended position. It will be appreciated that during rotation of thetrigger 4314 and movement of the plunger assembly 4310, the grip screw4308 remains static relative to the body of the firearm 4210F.

Turning now to FIGS. 54A-54C, a safety selector switch sensor 4400constructed in accordance to additional examples will be described. Thesafety selector switch sensor 4400 communicates with a safety switch4402 configured on the firearm. In the example shown, the safety switch4402 is shown configured on the long gun 4210A. It is appreciatedhowever that the safety selector switch sensor 4400 can be configuredfor use with any safety switch provided on any of the firearms discussedherein. In general, the safety switch 4402 is configured to move betweena safety position wherein actuation of the trigger is precluded (FIG.54A) and a first use position (FIG. 54B) wherein actuation of thetrigger is permitted. In examples, the safety switch 4402 canadditionally be configured to move to a second use position intermediatethe positions shown in FIGS. 54A and 54B. In examples, the first useposition shown in FIG. 54B can correspond to a fully automatic mode ofoperation of the firearm. An intermediate position can correspond to asemi-automatic mode of operation of the firearm. The configuration ofthe safety switch 4402 is merely exemplary and the safety selectorswitch sensor 4400 may be configured for use with any type of firearmsafety switch. The safety selector switch sensor 4400 communicates asignal to the event detection module 4200A indicative of a position ofthe safety switch 4402. The event detection module 4200A can use theadditional data from the safety selector switch sensor 4400 indicativeof the position of the safety switch 4402 to make a more accuratedetermination of an event occurring (e.g., a shot being fired). In otherexamples, the event classification signal 4450 can include additionaldata related to a status of the safety switch 4402. In this regard, thedetection module 4200 can communicate a status of the safety selectorswitch sensor 4400 in real-time to the server device 112 indicative ofthe safety switch 4402 being in the use or locked position. Suchinformation can additionally be downloaded and reviewed at a later timein a post mission analysis event.

With reference to FIG. 54C, a diagrammatic view of an event detectionmodule 4200 is shown. The event detection module 4200 represents any ofthe event detection modules 4200A-4200E described herein. The eventdetection module 4200 comprises a signal processing module 4430 having amachine learning module 4432 and a digital signal processing module4434. The sensors 118 output various sensor signals collectivelyidentified at 4440 to the signal processing module 4430. As discussedherein, the sensors 118 can include one or more IMU's each having atleast one of an accelerometer a gyroscope, a magnetometer, and ageolocation sensor. Additional sensors can be provided by the sensors118 and can include a pressure sensor, an audio sensor, a thermalsensor, combinations thereof, and the like. The trigger pull sensor 4300outputs a trigger pull sensor signal 4442 to the signal processingmodule 4430 indicative of the contact sensor 4320 of the trigger pullsensor assembly 4300 actuated to the activated position. The safetyselector switch sensor 4400 outputs a safety selector signal 4448 to thesignal processing module 4430 indicative of the indicative of the safetyswitch 4402 in the use position. The signal processing module 4430performs processing techniques to make decisions about an event based onthe signals 4440, 4442, 4448 and outputs an event classification signal4450. The event classification signal 4450 can be indicative of the typeof event identified. The event classification signal 4450 can be storedlocally on the event detection module and/or be communicated wirelesslysuch as the network 114. As explained herein, the type of event caninclude, but is not limited to, a shot being fired, a weapon maintenancethreshold being satisfied, a weapon failure being detected, anammunition type (live, blank) being detected or other event. Weaponmaintenance and/or weapon failure items can include a barreloverheating, or a barrel exceeding a shot count allotment. As willbecome appreciated, the signal processing module 4430 can makedeterminations based on some or all of the inputs 4440, 4442, and 4448indicative of the type of event identified. The more inputs that thesignal processing module considers will provide another layer ofconfidence in the type of event identified. With each additional layerof input consideration, false positives (such as falsely identifying ashot) can be reduced. In particular, the signal processing module 4430can use machine learning techniques and run algorithms that map a sensedevent to known events to determine if particular characteristics of thesensed event match or closely match known event patterns. In thisregard, various determinations about what exactly is occurring in thesample event can be made (e.g., a shot being fired, an ammunition typebeing fired, etc.). Machine learning technologies can train modelsexecuting the algorithms to continuously update known event signals tomore accurately represent a desired classification pattern. Inadditional examples, determinations can be made about the health of theweapon, the health of a user of the weapon or other environmentalcondition. The machine learning techniques can make determinations basedon analysis of one or more input signals. In this regard, the accuracyof event determination can be improved with each additional input layerconsidered (e.g., from additional sensors) by the model. In particular,the model can establish distinct thresholds that need to be met for eachinput layer for a given determination to be concluded (such as a shotbeing detected). In additional examples discussed herein, the eventclassification signals generated by the signal processing module can beadditionally used in training and performance applications. It isappreciated that in some event detection module configurations, some orall components may be located on the firearm and/or on a wearabledevice. For example, the event detection module 4200G shown in FIG. 2may incorporate the sensors 118 (including IMU) and signal processingmodule 4430 while the trigger pull sensor 4300 and safety selectorswitch sensor 4400 may be arranged on the firearm. In such examples, thecomponents of the event detection module 4200 can be configured towirelessly communicate as needed, such as by way of Bluetooth or othertechnology.

With reference now to FIGS. 55-56C, various methods and techniques fordetermining a classification of an event using digital signal processingtechniques will be described. In this example, the event classificationis a discharge event. In examples, the sensors 118 (e.g., an IMU) canprovide six signal inputs including three acceleration inputs (such asfrom one or more accelerometers) and three rotation inputs (such as fromone or more gyroscopes) measured on the firearm over a period of time.The inputs are diagrammatically shown relative to a firearm in FIG. 55 .The acceleration inputs can be provided as three accelerations in threedifferent directions. A first acceleration A1 is measured in a firstdirection along a first axis (labelled as x-axis), a second accelerationA2 is measured in a second direction along a second axis (labelled as ay-axis) and a third acceleration A3 is measured in a third directionalong a third axis (Labelled as a z-axis). The angular velocity(rotation inputs from the gyrometer of the IMU) can be rotations aroundthe respective axes. In particular, a first rotation R1 is measuredaround the x-axis, a second rotation R2 is measured around the y-axisand a third rotation is measured around the z-axis. In the example shownin FIG. 55 , the x-axis is generally defined transverse to the barrel,the y-axis is generally parallel to the barrel and the z-axis isgenerally along the grip of the firearm.

The acceleration and rotation signals can be received by the eventdetection module associated with the firearm. The following example willbe described in the context of the event detection module 4200A in thegrip module 4204A however it will be appreciated that the signalprocessing techniques can be used on any event detection module andfirearm such as those disclosed herein. The respective acceleration androtation inputs A1, A2, A3 and R1, R2, R3 are received at the eventdetection module 4200A. The event detection module 4200A uses machinelearning techniques that are trained on historical event determinationdata. The event detection module 4200A runs an identification algorithmusing the inputs to identify discharge event candidates and filters thecandidates to maintain only candidates having strongest likelihood torepresent a discharge event. In the example described, once anacceleration threshold is crossed, the acceleration and rotation inputsA1, A2, A3 and R1, R2, R3 are passed to a machine learning algorithmthat has trained on historical data. The machine learning algorithmcreates and runs identification algorithms to make a prediction (e.g.,shot detected) based on the similarity of the trained data (that isconfirmed to satisfy an event occurrence) with the sample datacollected. The machine learning models can be layered to map one or moresensed inputs 4440, 4442, 4448. As more models are satisfied by thesensed event a greater confidence can be made that the eventclassification output by the signal processing module 4430.

The methods and techniques described herein reduce the likelihood thatthe event classification signal 4450 is incorrect such as a falsepositive shot detection. In particular, while non-shot dischargingevents such as dropping a firearm or a firearm malfunctioning mayexperience acceleration and rotation inputs otherwise withinpredetermined thresholds that would classify as a shot being detected,the signal processing module 4430 can analyze additional signal inputlayers from the sensors 118 (such as from an audio sensor, pressuresensor and/or a thermal sensor, herein “other sensors”), the triggerpull sensor 4300 and the safety selector switch sensor 4400 to provideindependent systems that overlap the acceleration and rotation inputs.In this regard, while the primary inputs used by the machine learningalgorithms herein can be accelerations and angular velocities providedby the IMU of the sensors 118, the signal processing module 4430 canadditionally consider signals 4440 from other sensors of the sensors118, signals 4442 from the trigger pull sensor 4300 and/or signals fromthe safety selector switch sensor 4400 independently as supplementallayers of analysis. In examples, every machine learning system canexhibit holes where outputs can also include false positives.Complementary independent inputs, provided by additional sensors, canprovide further checks and balances in the confidence level of theultimate event classification signal 4450. By adding additional datainputs from systems independent from the acceleration and rotationinputs into the signal processing module 4430, these holes can beminimized and the confidence level of an accurate event classificationcan be improved, ultimately reducing false positives.

FIG. 56A is a first plot 4500 representing a combination of theacceleration inputs A1, A2 and A3 as a single directionless accelerationvector magnitude (y-axis) over time (x-axis, shown in milliseconds). Thefirst plot 4500 provides a plurality of sample event candidatescollectively identified at 4510 and individually identified at 4510A,4510B, 4510C, 4510D, etc. Each sample event candidate 4510 is associatedwith a respective window of time. The first plot 4500 can be used by thesignal processing module 4430 to classify events. In the first examplediscussed herein, signal processing module 4430 uses algorithms toclassify the event (or events) as a discharge event (e.g. a shotoccurring). The signal processing module 4430 can use algorithms toadditionally or alternatively identify a gas setting or an ammunitiontype being fired. In other examples, the signal processing module 4430can use algorithms to update a weapon status (e.g., maintenancerequired).

Each sample event candidate on the first plot 4500 is compared to a shotdischarge match filter. The shot discharge match filter can be astandard signal processing algorithm that uses a discharge accelerationtemplate that represents a confirmed weapon discharge event. Wherever asample event candidate matches or closely matches the dischargeacceleration template, the event is represented on a second signal plot4520 of dimensionless magnitude (y-axis) over time (x-axis). Otherfilters having templates specific to known event characteristics can beadditionally used to determine other event classifications.

The second signal plot 4520 can be generated that represents the matchesas spikes collectively identified at 4530 and individually identified at4530A, 4530B and 4530D. A confidence threshold line 4522 can beestablished where any spike that exceeds the confidence threshold line4522 can be classified as a discharge. Any spike that does not exceedthe confidence threshold line 4522 is eliminated as a candidate. Ofnote, in the example shown, the candidates 4510A, 4510B and 4510D allsatisfied the discharge acceleration template of the match filter andproceeded as spikes 4530A, 4530B and 4530D representing strongercandidates than the candidates shown in FIG. 56A. The candidate 4510Cdid not satisfy the discharge acceleration template and is not shown inFIG. 56B. In this regard, candidate 4510C is eliminated as a candidate.

Once the subset of accepted accelerations from the sampled eventcandidates that satisfy the discharge acceleration template have beenidentified, the rotation inputs R1 and R2 are considered. In thisregard, a first rotation input signal R1 from the sample window of timeassociated with each accepted acceleration is compared to a firstrotation template that represents a confirmed weapon discharge event. Inexamples, the first rotation input signal represents a rotation of thefirearm around the first axis x. As can be appreciated, during a typicaldischarge of a weapon, it is expected that the weapon will rotategenerally around the first axis x (in the direction R1 shown in FIG. 55) followed by a reverse rotation around the first axis. In this regard asecond signal processing step is conducted where the first rotationinput signal is compared to a first rotation template. If the firstrotation input signal is within the first rotation template, adetermination can be made that the sampled event candidate satisfies adischarge event.

In additional examples, a second rotation input signal R2 from thesample window of time associated with each accepted acceleration iscompared to a second rotation template that represents a confirmedweapon discharge event. In examples, the second rotation input signalrepresents a rotation of the firearm round the second axis y. As can beappreciated, during a typical discharge of a weapon, it is expected thatthe weapon will rotate generally in response to rotation of the bulletbeing fired. In particular, as the bullet rotates (from rifling of thebarrel) while translating within the barrel during discharge, thefirearm will have a resulting counter rotation. In this regard a thirdsignal processing step is conducted where the second rotation inputsignal is compared to the second rotation template. If the secondrotation input signal is within the second rotation template, adetermination can be made that the sampled event candidate satisfies adischarge event. As can be appreciated, further confidence isestablished as more layers of input signals are analyzed.

A third plot 4540 represents velocity (y-axis) over time (x-axis) of thespikes 4530 of the second plot 4520 (that have satisfied theacceleration and rotation filters described above). In examples, thespikes can be shown as traces collectively identified at 4550 andindividually identified at 4550A, 4550B, etc. within the sample windowcollapsed onto the third plot 4540. An upper threshold 4560 and lowerthreshold 4562 can be established by the signal processing module 4430that signify an upper and lower boundary of a profile consistent with ashot being fired. In this regard, any trace 4550 that follows within theboundaries of the upper and lower thresholds 4560 and 4562 can beidentified as a discharge event while any trace 4550 that goes outsideeither of the upper and lower thresholds 4560 and 4562 will bediscarded.

With reference now to FIG. 57 , various methods and techniques fordetecting a shot and for determining a shot count using machine learningmodeling techniques will be described. An exemplary logic flow of amachine learning technique for determining shot count is generallyidentified at reference numeral 4570. As described above, the sensors118, such as disposed on the firearm at the grip module can include anIMU that provides first, second and third accelerations A1, A2 and A3,and rotations R1, R2 and R3, respectively. The accelerations aremonitored constantly in real time and represent a plurality of sampleevent candidates. The first, second and third accelerations A1, A2 andA3 can be converted at convertor 4572 into an acceleration vectormagnitude 4576. The acceleration vector magnitude 4576 is directionlessand merely identifies a magnitude. In examples, the Pythagorean theoremcan be used to calculate the acceleration vector magnitude 4576. Asummer 4580 calculates a rolling sum of accelerations 4584 in apredetermined window of time. In the example shown, the summer 4580 usesten sample vector magnitudes per window. By way of example, sampling canoccur at 1125 Hertz (Hz) or slightly under a millisecond. As can beappreciated new windows having new samples are generated over time. Afirst new acceleration sample will be added to the summer 4580 while anoldest acceleration sample will drop off with each subsequent windowsample. In other words, the exemplary summer 4580 considers a rollingtotal of accumulative accelerations for ten samples and continuallyupdates to provide new sample windows each having at least some newsample accelerations.

The rolling sum output by the rolling sum module 4582 is compared to athreshold at a threshold evaluation module 4590. The threshold valuewill change for different weapon platforms. If the value of the rollingsum 4582 of the acceleration vector magnitude is below the threshold,the process loops to rolling sum module 4582 to obtain a new rolling sumfor analysis. If the acceleration vector magnitude is over thethreshold, the sample a candidate event is generated and passed to asample collector 4592. The sample collector 4592 collects raw dataincluding the accelerations A1-A3 and rotations R1-R3 from the IMU 118from the point at which the tier one threshold was crossed for apredetermined duration which is unique to each weapon system based onthe weapons cycle timing. In examples, for an M4, sixty samples arecollected, for an M2, eighty samples are collected. Other samples forother weapons may be collected. This predetermined duration can bereferred to as a candidate length.

The data from the sample collector 4592 is collected at a candidateevent module 4600. The candidate event module 4600 can assemble theacceleration and rotation data as candidate events over the amount ofdesired samples. In examples, the samples collected by the candidateevent module 4600 can be provided to an event alignment module 4604. Theevent alignment module 4604 can smooth data that may have overlappingentries, such as due to shots fired from automatic weapons. In examples,before the sample event is passed to the algorithm provided in themachine learning module 4432, the event is “fitted” to a standardizedacceptable waveshape for best match. This ensures that every shot eventis being passed to the algorithm in the same “position in time” as othershot events. This waveshape fitting is shown graphically in FIGS. 58A(before alignment) and 58B (after alignment) where an accelerationvector magnitude (Δv) measured by the IMU 118 is plotted over time(milliseconds). A baseline shot profile 4606 is shown having anacceleration vector magnitude Δv over time. The baseline shot profile4606 represents an expected shot profile of a known weapons type. Asample shot profile 4608A represents the profile generated by the signalprocessing module 4430 based on the signals 4440 received by the sensors118. The sample shot profile 4608A is shown also having an accelerationvector magnitude Δv over time. In the example shown, the baseline shotprofile 4606 and the sample shot profile 4608A both represent a shotfrom a fully automatic weapon. Notably, a residual acceleration spike4610A and a target acceleration spike 4612A are measured with the sampleshot profile 4608A. Similarly, a baseline acceleration spike 4614 isrepresented on the baseline shot profile 4606. The residual accelerationspike 4610A represents a residual force resolving from a previous shot.In examples, the residual acceleration spike 4610A can be created when abolt 4640 (FIG. 42A) from a bolt carrier group 4642 of the firearmreturns to a forward position. The target acceleration spike 4612Arepresents the force from the target shot. In this regard, the residualacceleration spike 4610A can be misleading data when attempting tocompare the sample shot profile 4608A with the baseline shot profile4606.

FIG. 58B illustrates the sample shot profile 4608B shifted or alignedwith the standard shot profile 4606. In particular, the algorithm of theevent alignment module 4604 shifts the sample shot profile 4608B to anearlier time such that the residual acceleration spike 4610B is movedout of alignment with the standard shot profile 4606. In this regard,the target acceleration spike 4612B is moved to an aligned position withthe baseline acceleration spike 4614. In the aligned position, a moreaccurate event threshold comparison can be made between the sample shotprofile 4608B and the baseline shot profile 4606. As such, thresholdsthat satisfy an event trigger are more accurately represented. Moreover,regardless of where an event is initially triggered based on thesethresholds, the event alignment module 4604 can properly aligncharacteristics of a sample shot with characteristics of a baseline shotthereby reducing error.

The accuracy of event detection is improved significantly as thealgorithm in the event alignment module 4604 is seeing events in thesame format for equal comparisons. Event alignment is particularlyuseful in weapons capable of automatic rate of fire such as, but notlimited to, an M4. For such weapons, overlapping data can exist from aprevious shot to a subsequent shot. In this regard, when using acumulative acceleration force threshold to trigger an event, the triggerfor an event can undesirably occur too soon because of an improperlycalculated cumulative force (considering force inputs outside of theproper window).

Referring to FIG. 57 , the samples are provided to a machine learningmodel 4630. The machine learning model 4630 receives the candidates asinputs and make determinations whether the sample candidate represents ashot occurring. In examples, the machine learning model 4630 can be aconvolutional neural network (CNN). The CNN can receive the candidatedata and predict whether the event is a shot or not. If the machinelearning model 4630 determines that a candidate satisfies the criteria,the candidate is output as a shot 4632. Conversely, if the machinelearning model 4630 determines that a candidate does not satisfy thecriteria, the candidate is output as no-shot 4634.

In examples, the CNN can be trained by a supervised training process.The CNN can be a traditional binary classification network where thedata from the sensors (such as the IMU) 118 is the input, and a targetis whether the data represents a shot. The training data can becollected on an as-needed basis to support model development. Inexamples, a model can generate predictions to form preliminary labels.The labels can in turn be reviewed, approved and/or modified by a user(human). In examples, a multiclass classification model can be used thatdistinguishes between ammunition types (blank vs. live rounds) oridentifying other key actions (such as reloading or a weaponmalfunction). The overall process of threshold-based event generationpassing to a CNN will remain similar to described above, however, theCNN will be multiclass and trained on more diverse data.

In still other examples, the sensors 118 can be configured to provideadditional sensors such as, but not limited to, a pressure sensor, ahigh-G sensor, a piezoelectric sensor, a resistivity sensor, and anaudio sensor. Such sensor inputs would be paired with the IMU data andpackaged into the input to the CNN. In one configuration two IMU's canbe incorporated having calibrations for different gravity sensitivities.For example, a first IMU can be configured to measure lower gravityranges such as +/−16G (32G's measured in 65536 steps). A second IMU canbe configured as a high-G accelerometer that can measure higher gravityranges such as +/−200G (400G's measured in 65536 steps). Otheraccelerometers can be incorporated that measure different ranges such as+/−400G. As can be appreciated, the high G IMU can be used todifferentiate weapon handling from weapon discharge. In other words, thedropping of a weapon or bumping the weapon on an object may create a 16Gsignal, but would be very difficult to also create a signal that issatisfied by the high-G sensor. When additionally incorporating an inputfrom a high-G sensor, an additional check and balance in the confidencelevel of the event classification signal 4450 is gained. Using theexamples described above, while non-shot discharging events such asdropping a firearm may experience acceleration and rotation inputs froma single low-G sensor within predetermined threshold that wouldotherwise classify as a shot being detected, the signal processingmodule 4430 can also consider another input layer such as accelerationand rotation inputs from a second high-G sensor. In this way, an eventsuch as dropping the weapon that would otherwise satisfy a thresholdprovided by the low-G sensor would not satisfy a threshold by the high-Gsensor. Such events can be rejected from further analysis improving theconfidence in the classification of the event. By way of example only,it may be easy to create a 16G signal from weapon handling, butdifficult to create a 150G signal from weapon handling. Similarly, itmay be easy to create a 150G signal from weapon discharge. In suchexample, the 150G acceleration can be either a discharge or a very hardimpact. The acceleration profile of the low-G IMU can used to determinewhether a discharge occurred.

In examples where the sensors 118 include a pressure sensor, a pressureis sensed in the immediate area of the firearm. As is known, when around is fired through the barrel of the firearm, the air pressure inthe immediate area changes. In examples, acceleration inputs from an IMUcan be analyzed as a first input layer and localized air pressure from apressure sensor can be analyzed as a second input layer. In exampleswhere acceleration signals resulting from a firearm being dropped orbumped that may otherwise (undesirably) reach a threshold indicative ofa shot being fired, a second input layer provided by a pressure sensorwould negate such false positive. In other words, even though anacceleration threshold has been met, a corresponding pressure thresholdwould not be met preventing a false positive shot detection.

In other examples where the sensors 118 include a pressure sensor andthe user of the firearm is a soldier, useful battlefield information canbe collected and acted on based on sensed pressures. In particular,explosions and other environmental pressure changes can be tracked andacted upon using information obtained from a pressure sensor. Pressuredata can be used for injury causation purposes, for example mildtraumatic brain injury (mTBI) or other blast reaction injuries may betraced back to a particular event where excessive pressure was measured.In other examples, situational awareness can be improved by collectingpressure sensor data from multiple users (soldiers in this example). Forexample, if multiple pressure signals indicative of a large blast ismeasured from a group of soldiers and received over the network 114,various responses can be deployed, such as by the responsiveinfrastructure 110 (FIG. 1 ).

With reference now to FIG. 42C, the sensors 118 can further includeposition sensors 4720 and 4722. In one example, the position sensorsinclude real time kinematics global positioning system (RTK GPS) sensorsor receivers 4720 and 4722. The receivers 4720 and 4722 can be disposedat locations creating a line that is axially parallel relative to abarrel (such as barrel 208, FIG. 2 ) of the weapon. The receivers 4720and 4722 can also be configured for use with global navigation satellitesystem GNSS using an international multi-constellation satellite system.In the example shown, the sensors 4720 and 4722 are generally disposedat offset locations on long gun 4210C. In examples, one sensor can bedisposed on the rail and the other sensor disposed on the butt stock ofthe firearm. It is appreciated that offset RTK GPS sensors can besimilarly positioned at spaced out locations relative to each other onother firearms. As is known, RTK GPS can be used to enhance the positionaccuracy of the Global Navigation Satellite Systems (GNSS). A first RTKGPS receiver can be configured as a “base” while the second RTK GPS canbe configured as a “rover”. The base receiver can transmit its positionand observed satellites data to the rover receiver. The rover can usethis information to exclude atmospheric errors. In examples, locationaccuracy to the centimeter can be achieved. Explained further, eachsensor 4720 and 4722 can use external signals (GPS, ultra-wideband asdescribed below, etc.) to determine its own location relative to theexternal emitter's location (GPS satellite, UWB beacon, etc.). Forinstance, with GPS, one sensor may determine its own position on Earth.One sensor, or both sensors 4720, 4722 can communicate their derivedposition to the other. Now both sensors 4720, 4722 know both locationsand can determine orientation. The sensors 4720 and 4722 can communicatewith each other in wired or wireless formats. A sample use case will nowbe described with a first sensor (“rear sensor”) 4722 disposed closer tothe buttstock and a second sensor (“forward sensor”) 4720 disposedcloser to the muzzle. The first sensor 4722 can determine its ownlocation using GPS. The second sensor 4720 can also determine its ownlocation using GPS. One of the first or second sensors can send itslocation to the other of the first and second sensors over wired orwireless connection. The first sensor 4722 now has two points on Earthand it knows the firearm 4210C is anchored between them. If the firstsensor 4722 looks at the second sensor's 4720 position and sees that itis 1 ft North and 1 ft East of the second sensor 4720, then it can bedetermined that the firearm 4210C is oriented at 45 degrees East ofNorth. In examples, the location and orientation information determinedby the sensors 4720 and 4722 can be communicated to the event detectionmodule 4200 and communicated to the network 114. Various parameters canbe tracked such as soldier weapon orientation, line of fire, threadlocation and situational awareness. In other examples, the weaponheading can be determined using the sensors 4720 and 4722 to determinelaunch angle of a long gun such as those described herein. In addition,the sensors 4720 and 4722 can be disposed on larger firearms includingmissile launching weapons for determining launch angles of largerprojectiles including missiles.

The sensors 4720 and 4722 communicate with a transmitter 4730 to provideenhanced positional data and heading orientation. In examples where thesensors 4720 and 4722 are configured as RTK GPS sensors, the transmitter4730 can be a satellite in orbit. In other examples, the sensors 4720and 4722 can be configured for communication with a ground basedtransmitter such as a beacon (such as beacon 2884, FIG. 30 ). Inexamples, the sensors 4720 and 4722 can additionally or alternativelycommunicate with ground based transmitters such as beacons inside of atraining facility. By way of example only, the beacons can be configuredto communicate signals in ultra-wideband format. In examples, thebeacons can be located at known static locations throughout the trainingfacility. In this regard, in such a soldier training environment, when asoldier enters the training facility, satellite tracking may be lost.The training facility can have beacons that can communicate with thesensors 4720 and 4722 to determine weapon orientation while inside thetraining facility. In examples, the sensors can dynamically switch fromcommunication to a satellite to communication to one or more localbeacons. The configuration of RTK GPS sensors 4720 and 4722 on a firearmprovides improved heading orientation determination over conventionalsensors such as magnetometers. It is contemplated that the sensors 118can include both RTK GPS sensors and a magnetometer. The event detectionmodule can be configured to alternate between using RTK GPS sensor dataand magnetometer data to determine an orientation of a weapon dependingupon the availability of an RTK signal.

In other examples, the CNN can be self-retraining for usercustomization. In such a system, a user begins with a generic modeltrained for their weapon system. The user populates the CNN withuser-specific shot data. The system would periodically retrain the CNNbased on their user-specific data to personalize the shot detection tobe best fitted to their individual firing technique. Improved insightscan be achieved into anomalous firing (user error, poor technique)compared to user baseline, which may provide the firer with morerelevant feedback. It is contemplated that a semi-supervised learningcycle where training data for a user-specific child model is initiallylabeled by the parent model, as the child model is fine-tuning the coreshot detection capability.

In other applications where the sensors 118 include an IMU, the signals4440 can include first, second and third accelerations A1-A3 and first,second and third rotations R1-R3 (see also FIGS. 54C and 55 ), thesignal processing module 4430 can be configured to make additionaldeterminations related to the mental and/or physical state of the user(e.g., user has fatigue, user is stationary, user is moving at a slowrate such as walking, user is moving at a fast rate such as running,user is moving at a very fast rate such as in a vehicle). In such anintegration, a machine learning algorithm can be trained on categorizeddata known to satisfy various states of the user. A sample datacollection over a period of time (also referred to herein as a sampleevent candidate), from the IMU in this example, would be compared ormapped to corresponding stored data associated with known or baselineevents. The signal processing module 4430 can make conclusions as towhat the current state of the user is based on the comparisons. In someexamples, the corresponding stored data can be generic to a collectionof users. In other examples, the corresponding stored data can bespecific to the user. In this regard, each user may have differenthabits or situational deployments that influence different data sets ofknown events. For example, a first user may have a first data set knownto represent running for the first user whereas, a second user may havea second data set known to represent running for the second user.

In yet additional examples, subsequent to making a determination of themental and/or physical state of the user, additional analysis can bemade related to shot performance. In other words, the accuracy of a shotcan be evaluated against a determined state of the user. Shot accuracycan be determined in any suitable manner such as post analysis of shotlocation relative to a target. In other examples, the event detectionmodule may receive feedback on shot location and subsequently evaluateshot performance based on the feedback. As such, further determinationscan be inferred as to whether a reduction in shot accuracy is related toa user's state (e.g., poor shot accuracy following high user fatigue).Various correlations can be made associating shot accuracy with thestate of the user firing the weapon. In additional examples, the sensors118 can additionally include data from sensors 120 of wearable devicesdisclosed herein. In examples, the sensors 120 can be one or morebiometric sensors that sense a biometric parameter of the user and senda biometric signal based on the sensed biometric parameter to the signalprocessing module 4430. The event detection module 4200 can determinethe state of the user based on the biometric signal. Biometric feedbackof the user (heartrate, temperature, etc.) can also be paired with thesensor output signal 4440 to add an additional layer of data analysis.

Returning now to FIGS. 58A and 58B, the algorithms provided in thesignal processing module 4430 can make additional determinations basedon the sample shot profile 4608B. The sample shot profile 4608 canrepresent actuation of a moving member in the firearm. By way ofexample, the moving member can include the bolt 4640 (FIG. 42A) from abolt carrier group 4642 of the firearm or a slide 4644 on a handgun(FIG. 55 ). While the following discussion is directed generally to abolt and bolt carrier group for a rifle the same principles can beapplied to other moving components on other weapons.

The acceleration spike 4612B can represent the bolt 4640 moving from aready position backwards when a round is initially fired (in a directionalong the second axis Y, FIG. 55 ) and an acceleration spike 4650B canrepresent the bolt 4640 subsequently moving forward and returning intothe ready position. In examples, the sample shot profile 4608B canrepresent a speed of the bolt carrier group 4642 during a discharge. Bytaking a large sample of multiple shots, a sample shot separation orspacing between subsequent shots for an automatic weapon can bedetermined and a resulting speed of the bolt carrier group 4642, andmore specifically a rate of fire can be determined.

FIG. 58C illustrates a signal or sample shot profile 4652 over a timewindow having three identifiable acceleration peaks including a firstpeak 4654, a second peak 4656 and a third peak 4658. The first peak 4654represents a shot fired event or when the shot is initially fired. Thesecond peak 4656 represents the bolt to rear buffer event or when thebolt 4640 hits the back of the buffer 4643 (FIG. 42A). The third peak4658 represents a bolt return event or when the bolt 4640 returns to theforward position. Explained further, the first peak 4654 represents anacceleration of the weapon measured by the sensors 118, in particular anacceleration along the A2 axis (FIG. 55 ) in a backwards direction(opposite a path of the bullet), caused by initial discharge of thebullet. The second peak 4656 represents an acceleration of the weaponmeasured by the sensors 118, in particular an acceleration along the A2axis in a backward direction caused by the bolt 4640 engaging the backof the buffer 4643. The third peak 4658 represents an acceleration ofthe weapon measured by the sensors 118, in particular an accelerationalong the A2 axis in a forward direction when the bolt 4640 returns to aforward, ready position.

A time between the second peak 4656 and the third peak 4658 representsbolt speed 4660. As can be appreciated, bolt speed 4660 is variablebetween shots and weapon platforms and is indicative of various weaponparameters such as gas pressure, part life and cycle speed. Such weaponparameters relate to an overall health of a weapon. The algorithmsprovided in the signal processing module 4430 can be trained withinformation that represents baseline bolt speed for any given weapon. Inthis regard, the machine learning and digital signal processing modules4432 and 4434 can make comparisons of any shot profile, such as the shotprofile 4652, with known/expected shot profiles and make determinationsrelated to weapon health. For example, if a measured bolt speed 4660 islonger than an expected bolt speed (such as beyond a predeterminedthreshold time), the signal processing module 4430 can output an eventclassification signal 4450 indicative of an insufficient bolt speed thatrequires a maintenance action. In examples, a notification can be sentto the user 202 such as to a wearable device of the user 202 (e.g., theearpiece 222, the eyeglasses 224).

The algorithms provided in the signal processing module 4430 can makeadditional determinations related to weapon operational status based onthe sample shot profile 4652. In particular, the signal processingmodule 4430 can determine an operational status of the firearm based onanalysis of the first, second and third peaks 4654, 4656 and 4658. Insome examples, the second and/or third peak 4656, 4658 may be missing orundetected from a sample shot profile 4652 indicating a failed weapon orfailed shot operational status. In examples, the sample shot profile4652 can be compared to a baseline shot profile representative of aproperly functioning firearm. The baseline shot profile can include afirst peak at a first time representing initial discharge of thefirearm, a second peak at a second time representing a bolt engaging arear buffer of the firearm, and a third peak representing the boltreturning to the forward position, as discussed above.

In one example, if the firearm is under gassed or not enough of theexpanding gasses are passed back to the bolt assembly 4642, the rearwardmomentum of the bolt assembly 4642 is insufficient to overcome thepushback from friction and the buffer spring, so the bolt assembly 4642never reaches the full rearward position. In such a scenario, the secondpeak 4656 is marginalized, absent entirely, or otherwise insufficientfrom the sample shot profile 4652. An event classification signal 4450can be sent indicative of an undergassed weapon that requires amaintenance action. In examples, a notification can be sent to the user202 such as to a wearable device of the user 202 (e.g., the earpiece222, the eyeglasses 224).

In another example, if the last round in a magazine is discharged, thebolt 4640 can be jammed or lock to the rear at the buffer 4643. In thisscenario, because the bolt assembly 4642 never returns to the forward,ready position, the third peak 4658 is insufficient or absent entirely.An event classification signal 4450 can be sent indicative of a jammedweapon that requires a maintenance action. In examples, a notificationcan be sent to the user 202 such as to a wearable device of the user 202(e.g., the earpiece 222, the eyeglasses 224). Notably, absence of thethird peak 4658 does not represent the absence of a discharge event asthe third peak 4658 is not an inextricable trait of a shot event.

FIGS. 59A-59C represent baseline shot separations 4700, 4702 and 4704for new, medium and old firearms, respectively. The algorithms in thesignal processing module 4430 can have predefined ranges for shot countsconsistent with shot counts indicative of a new firearm, a mediumfirearm and an old firearm.

As shown, as a firearm is used over its life, it will have aprogressively faster rate of fire. In this way, a medium aged firearmwill have a faster rate of fire compared to a new age firearm.Similarly, an old age firearm will have a faster rate of fire comparedto a medium and new age firearm. As a fully automatic weapon ages andexperiences more use, its rate of cycling through rounds becomes faster.Rate of fire can be used to track and monitor weapon performance. InFIGS. 59A-59C, an event window is 60 samples long meaning the shotscannot be less than 60 samples apart. The example shown is for an M4rifle having a cycle rate of about 700-950 rounds per minute. Atabsolute maximum rate of 950 rounds per minute, each shot should take 63milliseconds. As the samples are taken at 1125 Hz, 71 samples arerepresented. At minimum rate of acceptable fire, 96 samples arecollected between rounds.

In examples, the algorithms in the signal processing module 4430 cancompare the measured shot separation of a sample shot profile 4608B tothe baseline shot separations 4700, 4702 and 4704 to determine anoperational status of the firearm. In examples, a shot separation can bedefined as a distance between like events in subsequent shots. Inexamples, a shot separation can be defined as a time between a firstpeak 4654 of the sample shot profile 4652 and a subsequent first peak ofa subsequent sample shot profile. In additional examples, maintenancestatus can be determined. It is contemplated that any sample shotprofile that is outside of an acceptable shot separation from any of thebaseline shot separations 4700, 4702 and 4704 can be flagged forpreventative maintenance (e.g., service and/or replacement of acomponent such as a barrel), or a determination can be made that thefirearm has reached a target lifespan and needs to be replaced entirely.In examples, a preventative maintenance alert or message can be outputby the event detection module 4200 with the event classification signal4450 for consumption by the user. It is contemplated that suchpreventative maintenance steps can be conveyed in real time or postoperation after an event has completed.

In other examples, the algorithms in the signal processing module 4430can compare the measured shot separation of a sample shot profile 4608Bto the baseline shot separations 4700, 4702 and 4704 to determine gassettings on a firearm. As is known, in a gas-operated rifle, a portionof high-pressure gas from the cartridge being fired is used to power amechanism to dispose of the spent case and subsequently reload a newcartridge into the chamber. Improper gas settings can degrade the healthand performance of a weapon. It is contemplated that any sample shotprofile that is outside of an acceptable shot separation from any of thebaseline shot separations 4700, 4702 and 4704 can be flagged formaintenance of the gas setting.

As mentioned above, the shot detection techniques can be furtherleveraged for subsequent training and performance analytics. The signalprocessing module 4430 can be configured as a shot timer for determininguser performance such as split times between discharges. The split timescan be used toward qualifying metrics for performance accuracy. Inexamples, split times from multiple users (law enforcement, military,etc.) having event detection modules 4200 configured on their firearmare determined and compared with each other. For example, in someinstances, training exercises or drills can be carried out whereparticipants may be all engaged concurrently.

Each event detection module 4200 detects discharge and split times onshots distributed across multiple users all engaged at once. Inexamples, the shot times can be aggregated for an entire squadparticipating in a particular drill. In other examples, the split timescan be analyzed and compared to rank how each user of the squad isperforming compared to other users of the squad in the drill. In furtherexamples, split times can be linked to a set of scenario parameters(e.g., target location relative to shooter, shooter mobility betweenshots (a distance calculated between a first location the user is atwhen the first shot is detected and a second location the user is atwhen the second shot is detected), presence of other shooter).Conclusions may be made that may tie split times to the scenarioparameters to determine whether a particular parameter or parameters maybe influencing split times.

In traditional training settings where multiple individuals (policeofficers, soldiers, etc.) are participating concurrently it can bedifficult to accurately access individual performance and make anycorrelations between results and a particular individual. An overallassessment can be made to compare how certain individuals are performingin comparison to other individuals participating in the same trainingexercise or drill. Additionally, an overall aggregated assessment can bemade on a group (squad, a platoon, a company group etc.) as a wholewhereby the collective performance of an entire group may be compared toanother group conducting similar drills. Additional performance metricsbeside split times can be determined by the event detection module 4200using the shot detection techniques described herein. Such metrics caninclude transitions and elapsed time. Transitions can be a durationbetween shots on distinct targets. Elapsed time can be a time from astart signal and the final shot. A start signal can be input by one ofthe sensors 118 that detect an external signal indicative of a start ofan event.

Referring now to FIG. 60 , an exemplary method determining performancemetrics using the shot detection techniques of the event detectionmodule 4200 is shown and generally identified at reference numeral 4750.At 4752, shots are detected at multiple event detection modules formultiple users. At 4754, split times are determined for each user basedon a time lapse between subsequent shot detections of each user. Inother words, for a single user, a first shot discharge is detected at afirst time, a second shot discharge is detected at a second time and adifference in time is determined as a split time. This same strategy iscarried out for multiple users in the event that comparisons betweenusers and/or groups of users are being made. At 4760, split times areassociated with various scenario parameters. By way of example, scenarioparameters can include target location relative to shooter, shootermobility between shots, and presence of another shooter. At 4764, splittimes are compared between users or groups of users. In examples, splittimes can be averaged by group and compared with corresponding splittimes averaged by other groups. At 4770, a performance score or rankingfor each user or group is determined based on the comparing. Inexamples, split times determined for various event detection modules canbe communicated to each other where comparisons are determined. In otherexamples, split times can be communicated wirelessly to the network 114where application 102 can execute and output the calculations andcomparisons/rankings.

In additional examples, a stability index can be determined based on auser's weapon orientation before, at, and subsequent to dischargedetection. With additional reference to FIG. 61 , a method of tracking astability of a firearm during a discharge event is shown generally atreference numeral 4800. At 4808, the event detection module 4200receives IMU inputs over a timeframe. At 4810, the event detectionmodule 4200 determines a discharge event within the timeframe. Dischargedetection may be determined by any of the methods described herein suchas, but not limited to, the signal processing and machine learningalgorithms associated with FIGS. 56A-58B.

A stability index can be determined based on how stable a shooter isholding the weapon immediately before and immediately after a shot.Weapon stability can be determined using the signals from the IMUincluding the rotation inputs R1, R2 and R3. As can be appreciated, astability index can provide insight as to a user's ability to controlweapon behavior surrounding a shot (recoil, etc.). A stability index hasimplications toward barrel tracing for a particular shot. Barrel tracingcan be generally defined as a first orientation of a barrel (R1, R2, R3inputs at the first time) immediately prior to pulling of the triggercompared with a second orientation of the barrel (R1, R2, R3 inputs at asecond time) and immediately after pulling the trigger (R1, R2, R3inputs at the third time). Insights can be made to determine how a useris anticipating a shot and reacting to a recoil subsequent to the shotbeing fired. In other words, barrel tracing can track the movement ofthe weapon in space at a timeframe taken from a first time immediatelyprior to a shot detection, at a second time at the instant of shotdetection, to a third time immediately subsequent to a shot detection.At 4814, a first orientation of the firearm immediately prior to thedischarge event is determined. At 4820, a second orientation of thefirearm at the discharge event is determined. The orientation of theweapon at the first time (prior to discharge) can be compared to theorientation of the weapon at a second time (at discharge, also referredto as target or dead-center orientation). This first comparison can giveinsights as to how the user may be compensating weapon orientationanticipating the shot. At 4822, a third orientation of the firearmimmediately after the discharge event is determined. The orientation ofthe weapon at the second time (at discharge) can be compared to theorientation of the weapon at a third time (after discharge). This secondcomparison can give insights as to how the user has reacted to a shotoccurring. At 4826 a stability index is assigned based on at least twoof the first, second and third orientations.

In examples, barrel tracking data can be converted into a numberrepresenting the stability index. The stability index can be a simpleinteger that is easily comparable between various weapon users.Furthermore, shot accuracy can be correlated to a correspondingstability index to make associations between the shot accuracy and thestability index. As can be appreciated, the IMU data may show a userwith less control of their weapon leaning into a shot or undesirablycompensating for a shot leading to a lower score stability index and areduced shot accuracy. Real time feedback can be provided to the user at4828.

The event detection module 4200 can determine barrel tracing and astability index using inputs from the sensors 118 including an IMU. Forexample, and as described above an IMU can provide first, second andthird accelerations A1, A2 and A3, and rotations R1, R2 and R3,respectively (see also FIG. 55 ). Movement and orientation informationof the barrel before, during and after pulling the trigger can bemeasured. Determinations can be made of how a user (law enforcement,soldier, etc.) is anticipating a shot (IMU measurements takenimmediately prior to a shot detection) and how the user is reacting tothe shot (IMU measurements taken immediately after a shot detection).This information can be communicated back to the user such as throughthe usage monitoring system 2800 (FIG. 29 ) including augmented reality(AR) or virtual reality (VR) headset or goggles (described in moredetail herein). In this regard, real time updates can be provided in anintegrated visual display that would show exactly how and to what degreethe user is moving (leaning, etc.) that may adversely impact stabilitybefore or after a shot. Using this information and feedback can allowfor subsequent and quick correction and improvement of lethalitymetrics. In particular, by providing real-time feedback to the user, auser can make corrections immediately upon learning of their barreltracking data. In this regard, corrections can be implementedimmediately, such as after one to ten shots rather than after an entiretraining exercise or other mission where more than ten, or hundreds ofshots have been fired. Corrections can therefore be implemented by theuser in real-time improving shot accuracy and lethality metrics as awhole.

It is contemplated that orientation data for a particular user can betracked over time and be used in various algorithms at the machinelearning module 4432 or digital signal processing module 4434 toestablish user profiles and tendencies as they relate to performanceevaluation. In examples, a historical baseline stability index of aparticular user can be established based on collected historicalstability index information about the first user. Progress of shottendencies can be tracked to give the user an understanding ofperformance improvement (or decline). In other words, a particular usercan have plurality of historical shot data stored at the signalprocessing module 4430 where the machine learning module 4432 canestablish a baseline of that user. In this regard, any subsequent shotcan be compared to a user's established historical baseline performanceto determine how the evaluated shot compares to a user's typical(baseline) shot. In examples, more tailored recommendations can be madespecific to a particular user rather than an evaluation that couldotherwise yield a generic predetermined recommendation based on a largedatabase of other shooters. Such comparisons can be displayed to theuser in real-time or post mission during subsequent analysis.

Recommendations can include any suggested action that may improve weaponstability. For example, signal analysis may reveal that a user istipping the barrel of the weapon downward just before shot detection. Arecommendation may be made to level the barrel prior to pulling thetrigger. Similarly, a signal analysis may reveal that the barrel of theweapon is tipping upward just after shot detection. A recommendation maybe made to hold the weapon steadier to compensate for recoil. As isknown, allowing the weapon to rotate upwardly due to recoil can haveadverse consequences such as, but not limited to, an inability to beready for a subsequent shot. Recommendations can be communicated (video,audio, haptics) to a user such as at the usage monitoring system 2800including audio and visual headsets, computer displays, etc. Inexamples, live recommendations can be communicated as recommendations(for example displayed on an AR/VR display 5400 disclosed herein withrespect to FIG. 70 ) to the user during use (such as a trainingexercise). By making real-time feedback and/or suggestions a user canquickly make adjustments to improve any performance or lethalitymetrics. In additional examples, the systems described herein can beconfigured to communicate a shot metric related to shot accuracy. In oneexample, smart targets can be included that provide feedback of shotaccuracy. Smart targets can have sensors that determine a location of adischarge on the target (e.g., impact location of the projectile). Othersmart targets can be configured to utilize computer visual analysis(such as on a paper target) to determine location of projectile impactand correlate the determined impact location with an accuracy metric. Inexamples, the smart targets can communicate a shot accuracy metricsignal based on an accuracy of the shot wirelessly (Bluetooth, Wi-Fi,etc.) to an event detection module 4200.

It is contemplated that any user feedback related to any performancemetric may additionally include information related to where the shotwent relative to a target. Such results may be incorporated in anyreal-time feedback examples or subsequent review such as on a computerdisplay. In other examples, an image can be taken of holes on a papertarget that has been shot through during a training exercise. The holescan be identified such as by computer imaging processes. In examples,the image can be pushed to the network 114 where calculations andanalysis can take place such as at application 102. Distances between acentral target and the holes can be measured to comparatively assessshot accuracy. In still other arrangements, by using the accelerationsand rotations from the IMU, and various environmental conditions, thesignal processing module 4430 can calculate a projected bullet (“virtualbullet”) path and destination. In examples where indoor shootingexercises are carried out UWB emitters 4730 cooperating with the sensors4720 and 4722 can provide precise weapon orientation whereby bullettrajectory and final destination can be estimated. It is contemplatedthat many variables can be included for the environmental conditionsthat may impact bullet trajectory and destination. Furthermore, it iscontemplated that a projected bullet path and destination can beestimated based on either live fire or blanks.

Shot accuracy can be communicated back to the user to assessperformance. In this regard, the techniques described herein candetermine performance metrics such as split times, barrel tracing andstability indexes. These performance metrics can be mapped to shotaccuracy metrics to provide a full picture of how such performancemetrics may influence shot accuracy.

Referring to FIG. 62 , a weapon usage monitoring system according toadditional embodiments is shown and generally identified at referencenumeral 5200. The weapon usage monitoring system 5200 generally includesvarious weapon response devices associated with users or soldiers. Asused herein “weapon response device” can include at least the eventdetection module 4200 described above. In the example shown, a firstweapon response device 5210A is associated with a first user or soldier5212A, a second weapon response device 5210B is associated with a seconduser or soldier 5212B and a third weapon response device 5210C isassociated with a third user or soldier 5212C. It will be appreciatedthat fewer or additional weapon response devices may be provided withwhatever quantity of users is necessary. In embodiments, each weaponresponse device may produce a weapon heading, discharge detection,and/or a gesture detection based on at least an IMU/motion sensor asdescribed herein. Weapon response data 5214 associated with all of theweapon response devices 5210A, 52106 and 5210C can be communicated tothe cloud 5216 in real time through a communication device or networkhub (such as an EUD 5218 or software defined radio (SDR) 5220) such asby Transmission Control Protocol/Internet Protocol (TCP/IP). It isappreciated that the network hub (SDR 5220) can communicate by anysuitable method. As used herein the cloud 5216 can be a cloud basedcomputing interface device that can aggregate and interpret weaponresponse data. A weapon response device may be ‘local’ (˜2m) to the arearegister with each other via intra-soldier wireless (ISW) handshakeauthentication. A weapon response device may communicate user and weapondata to other registered weapon response devices in the area via ISW.Weapon response device measurements may be pushed to localvisualizations 5222 (e.g., AR/VR or EUD) or to the cloud 5216 through asingle TX point such as an SDR or EUD (mobile phone) 5218. Additionallyor alternatively, the user and weapon data may be communicated to asecondary device 5230 by way of TCP/IP. In examples, the secondarydevice 5230 can aggregate the weapon response device data from therespective weapon response devices 5210A, 5210B and 5210C.

In examples, the AR/VR display 5222 can be a wearable device that a userin the field (soldier or other field personnel) can visualize actions(and/or directional engagement information from other soldiers) from theweapon data. In some embodiments a user wearing such AR/VR display 5222can see through obstacles (walls, etc.) to understand the layout of aroom or space (e.g., including potential enemies and threats). Thefirings of rounds of weapons will be viewable by the AR/VR display. Itis further contemplated that such scenarios may be used in trainingexercises to simulate various battle circumstances.

Referring to FIGS. 63-65 , in embodiments a weapon response devicesensor 5232A of the weapon response device 5210A may be integrated intothe grip or rail of a rifle, or the grip of a pistol (e.g., a Glock17/22), or some other weapon location. While one weapon response devicesensor 5232A is shown associated with the weapon response device 5210Aused by the soldier 5212A, it will be appreciated that all weaponresponse devices disclosed herein will include an associated sensor. Aweapon response device sensor 5232A may collect time-series IMU data5240 from a weapon's usage. IMU data may be saved as raw data as well asprocessed ‘events’ that are classified, for example, by AI algorithms5242 on hardware. Data 5244 may be saved in flash memory on the weaponresponse device, with no RF broadcasted and no network connectionrequired. Data may be batch offloaded back at a central location, suchas an armory/headquarters via Bluetooth connection to a mobile device5248, which may upload to the cloud 5216 such as by way of a cloud APIHTTP POST data upload 5249. Data may be reviewed by an end user, forexample, via a web browser to ‘recreate/replay’ the engagement from thecollected/recorded data. Backends and user interfaces may be designedfor large scale engagements (e.g., 60+ users of viewable data). Inexamples, at 5250, AI algorithms classify additional events as holster,unholster, active aim, stability index and discharge. At 5252, AIalgorithms aggregate data from multiple devices to determine threatlocation data (triangulation of weapon heading vectors). At 5254, raw,unprocessed data from a weapon response device or other sensors iscollated. At 5260 an event catalogue is produced. The event cataloguecan include data saved to event catalog that organizes classified eventsby user, timestamp, event type, location, etc.

Referring to FIG. 65 , an exemplary replay implementation system andmethod according to various embodiments is shown and generallyidentified at reference numeral 5270. A saved user workspaces module5272 can include user specific settings and saved segments of weaponresponse device data of interest within a user account. A permissionsand access controls module 5274 can dictate user permissions and datarestrictions for viewing and replaying saved data. Various end users arecontemplated such as a criminal justice investigator 5280, a departmentof defense engagement personnel 5282 and an AI gaming developer 5284.End user cases can include training, investigations for criminaljustice, tactics analysis, debrief, criminal evidence, and gamingalgorithm input or training data.

Referring to FIG. 66 , in embodiments a weapon response device 5210A mayimplement STANAG 5740 and AEP-90 interface (Standard I-Rail™ interface),mounted on a TWORX™ I-Rail™. A weapon response device 5210A may produceweapon heading, discharge detection, gesture detection, or some otherdatum, based on, for example, IMU/motion sensor. Measurements may bepushed from the rail mounted weapon response device through anintermediate hop 5300 (e.g., Bluetooth/phone 5302, Intra-soldierwireless ISW/software defined radio (SDR) 5304, ISW/I-Rail™)Measurements may also be saved locally to the device, such as in flashmemory 5306 in RF denied/sensitive environments. Weapon response devicemeasurements (and other sensors, such as soldier borne sensor SBS 5310that communicate to/through the weapon response device 5210A) may bepushed to the cloud 5216 (e.g., a cloud associated with the weapon usagemonitoring system) for aggregation, analytics, or some other processing.Processed results may be pushed back to the mobile device 5230 orbrowser, or other facility, for viewing/reviewing in real-time,near-real time, and/or post-mission debrief.

Referring to FIGS. 67 and 68 , sensor hub aggregation examples areshown. FIG. 67 illustrates a first implementation including a platformsensor hub aggregation, rail or I-Rail™. FIG. 48 illustrates a secondimplementation including a grip mounted sensor hub aggregation. Inembodiments soldier borne sensors (SBS) 5310 may send information to theexternal sensor 5320 such as a Rail Control Module (RCM) via ethernet,on the Intelligent Rail (I-Rail™)/Power Data Rail, and/or of some othertype of power rail, data rail, or power/data rail combination. A weaponresponse device 5210A may interface with, for example, an externalsensor 5320 (such as an RCM module) through at least one of thefollowing methods: direct Connection via serial connection or datawire/I2C; I-Rail™ 5324 such as an I-Rail™ wired network connection viaethernet; I-Rail™ wireless network connection: through the I-Rail™ viaethernet, ISW, WIFI, or Bluetooth connection. The weapon response device5210A may receive SBS information through the external sensor 5320interface connection. The weapon response device 5210A may aggregate theadditional SBS information into threat location modeling and datavisualizations in ATAK or AR/VR. The weapon response device 5210A maypass additional SBS information to other intelligent systems on thesoldier via ISW wireless link. The weapon response device 5210A cancommunicate measurements and/or data to an integrated visualaugmentation system (IVAS) 5332. Further to the above examples, the IVAS5332 can display information live metrics in real-time related toperformance metrics including barrel tracing, stability indexinformation, split times, ranking versus other individuals or squads,comparisons of recent shot activity to a user's own baseline shotprofile. Additionally, the weapon response device 5210A can communicatemeasurements and/or data to an end user device EUD 5340, such as amobile phone. The EUD 5340 can then communicate the measurements and/ordata to the cloud 5216.

In embodiments, data relating to weapon heading and/or discharge frommultiple users may be pushed to the cloud (e.g., a cloud associated withthe firearm usage monitoring system). The intersection of multipleweapon heading vectors may be calculated in the cloud (e.g., a cloudassociated with the weapon usage monitoring system). Additional sensordata may be fused with weapon heading vector intersection point for acombined multi-sensor fused solution. Error across a plurality ofsensors and weapon heading(s) may be combined using filtering/sensorfusion to produce an estimated error ellipsis that contains the target.An intersection point may be provided through the cloud (e.g., a cloudassociated with the weapon usage monitoring system) API to variousvisualizations.

Referring to FIGS. 69 and 70 , in embodiments a weapon response device5210A may produce weapon heading, discharge detection, gesturedetection, or some other data, based at least in part on IMU/motionsensor. A weapon response device 5210A may be trained to recognizedischarges vs. “false positives” such asmalfunctions/explosions/dropping and the like as described herein.Measurements may be saved to flash memory 5306 local to the device.Power may be optimized to survive long time periods (e.g., 60 days; 72hours with GPS) in certain configurations. Upon return to a centrallocation 5360, such as an armory, the weapon/weapon response device(such as weapon response devices 5210A, 5210B, and 5210C) may be‘checked-in’ and scanned. Scanning may offload all saved information onthe respective weapon response devices 5210A, 5210B and 5210C. Data maybe used for training, replay, simulation, weapons maintenance, or someother purpose. In examples, the data may be offloaded onto a personalcomputing device 5370 and uploaded to the cloud 5216.

Referring to FIGS. 70 and 71 , in embodiments a weapon response device5210A may produce weapon heading, discharge detection, gesturedetection, or some other data, based on an IMU/motion sensor.Measurements may be pushed from the rail mounted weapon response devicethrough an intermediate hop (Bluetooth/phone, ISW/software definedradio, ISW/I-Rail™). Measurements may also be saved locally to thedevice in RF denied/sensitive environments. Weapon response devicemeasurements (and other sensors that communicate to/through the weaponresponse device) may be pushed to the cloud 5216 (e.g., a cloudassociated with the weapon usage monitoring system 5200) for aggregationor analytics in either real time, near real-time or post mission during,for example, a batch offload process that may be initiated by the user.

In additional examples, the event detection module 4200 (FIG. 54C) ofthe weapon response device 5210A can be configured to include a tippingsignal with the event classification signal 4450. A tipping signal ornotification can be used to tip-off or alert other infrastructure orsystems (e.g., other resources such as orbital resources including asatellite, UAV or other manned aircraft, etc.) to perform a response oran action based on an input received by the sensors 118 (accelerationand rotation signals generated from IMU's as discussed herein). In thisregard, the event classification signal 4450 acting as a tipping signalcan be indicative of any operational status of the firearm as discussedherein such as shot detection, weapon heading, threat detection, soldierhealth status, etc.

To this end, the tipping signal 4450 may be pushed to the cloud 5216 foraggregation in real time where a responsive action from the responsiveinfrastructure 110 (FIG. 1 ) can be initiated (e.g., a UAV deployed tothe deployment location such as to provide enemy engagement supportand/or replenish ammunition and/or replenish manpower) based on receiptof the tipping signal. As discussed further herein, the tipping signal4450 can be received initially by a Local Area Network (LAN) 5512provided by the mobile networking hub 5510 (e.g., a Wi-Fi router) or auser-provided short- to mid-range network when direct communication withthe cloud is unavailable. The tipping signal 4450 can be any operationalstatus event of the firearm such as, but not limited to, a dischargeevent.

The UAVs or other aircraft may be configured to drop ammunitionre-supplies within the deployment location, for example, in response tothe tipping signal 4450 and/or the application 102 determining thatcurrent ammunition supplies of one or more users of firearms 104 arerunning low or depleted before, during, or after an engagement with adetected threat. In another example, response infrastructure 110 may beor include transport vehicles used to transport reinforcements withinthe deployment location, for example, in response to application 102determining that additional manpower is required or would be beneficialfor engaging the detected threat based in part by the tipping signal4450.

In embodiments, response infrastructure 110 may refer to components,assets, or other matter rather than to specific infrastructure used totransport or otherwise deploy those components, assets, or other matterwithin the deployment location. For example, response infrastructure 110may refer to firearms, ammunition, medical equipment, or other assetswhich can be deployed using a UAV, another aircraft, or another deliverymechanism. In embodiments, response infrastructure 110 may refer tolocations, components, assets, or other matter which may not travel tothe deployment location. For example, response infrastructure 110 mayinclude or otherwise refer to one or more locations at which assetinventories (e.g., firearm, ammunition, medical, or other inventorystocks) are stored and/or to hardware or other machinery or assets atthose locations.

In additional examples, the weapon response device measurements can becommunicated to the soldier by way of augmented reality/virtual reality(AR/VR) 5400. Weapon response device data from a plurality of users maybe visualized for: individual users for real time feedback andsituational awareness (e.g., round count, friendly location, weaponheading, and the like); and weapon response device data may be capturedand provided back to a user in augmented reality/virtual reality (AR/VR)5402 for training, simulation of scenarios, or some other purpose.

In embodiments, a weapon usage monitoring system 5500 may include aweapon response device that is deployed to a variety of environmentswith a plurality of RF characteristics, requirements, and profiles. Inthe example shown in FIG. 71 , weapon response devices 5212A, 5212B,5212C, 5212D and 5212E are illustrated in an environment withoutinternet connectivity. A mobile networking hub 5510 provides a localarea network 5512. Each of the weapon response devices 5212A, 5212B,5212C, 5212D are configured to communicate respective weapon responsedata (such as described above) to the mobile networking hub 5510. Inembodiments, long range RF communications that provide connectivity tothe general cloud or internet might not be feasible, possible, or arenot desirable in certain deployment conditions. In cases such as these,the weapon usage monitoring system 5500, as described herein, mayprovide a disconnected networking solution that provides near-real timesituational awareness data on a local area network with no internetconnectivity required. This network may provide communications within asquad or within a local area network via mid- or short-range networkcommunications, allowing streaming data from weapon response devicesensors, as described herein, to be processed and consumed on air-gappedor disconnected networks with no internet connectivity, as shown in FIG.71 .

In embodiments, a weapon usage monitoring system 5500 may include aweapon response device such as any of the weapon response devices 5212A,5212B, 5212C, 5212D that is within the grip of a weapon, the picatinny(or similar) rail, or the power data rail of a weapon. The weaponresponse device may collect data from the IMU, or other sensor, andprovide machine learning classifications of the data running locally onthe weapon response device. Classified events, as well as the full datastream, may be pushed via Bluetooth, ISW, or similar wireless format toan end user device such as respective end user devices 5610A, 5610B and5610C. The end user device can be a mobile phone, or other clientdevice, paired with the weapon response device, for example in a 1:1mapping. The mobile phone, mobile computing device, or other clientdevice may run additional services that ingest this data. The servicesrunning on the mobile phone or other client device may process the dataand produce additional machine learning event classifications from thedata that may have been too computationally intensive to run on theweapon response device itself locally.

In embodiments, events that were provided by the weapon usage monitoringsystem's machine learning algorithms running on the weapon responsedevice and the end user device may be combined with the raw IMU, orother sensor data produced by the weapon response device and pushed ontothe Local Area Network (LAN) 5512 provided by the mobile networking hub5510 (e.g., a Wi-Fi router) or a user-provided short- to mid-rangenetwork. The mobile networking hub 5510 may travel with a squad ofusers, and may be deployed within a vehicle, personnel (such as forexample on a backpack or other wearable), autonomous vehicle (drone,etc.), or other mobile resources localized within the squad. Thenetworking hub may provide networking required administrative operationssuch as discovery protocol, dynamic host configuration protocol (DHCP),or other protocols. The networking hub may also provide connectivity tolocally deployed edge compute resources 5560.

In embodiments, the weapon usage monitoring system 5500 may include edgecompute resources 5560 that consist of, at least in part, squaddeployable or wearable compute resources that provide additional,centralized and localized processing power for a final layer of machinelearning, squad-based insights on aggregated data. In examples, the edgecompute resources 5560 aggregates weapon response data from the mobilenetworking hub 5510 into aggregated weapon response data from all of theweapon response devices (5210A, 5210B, 4210C, etc.) It is appreciatedthat all of the communications between all of the weapon responsedevices to and from the mobile networking hub 5510 and the edge computeresources 5560 occurs on-site and within the (self-contained) LAN 5512where no traditional internet connectivity is available (or desirable toconnect with). In an example, edge compute resources 5560 may physicallyconsist of one or more AWS snowball instances, but may also be deployedto a variety of edge deployment infrastructures that supportvirtualization. The edge compute resources 5560 of the weapon usagemonitoring system 5500 may allow for a localized, edge based virtualcloud/cluster infrastructure that aggregates data from weapon usagemonitoring system-based and/or weapon response device-based sensors andedge compute resources to provide a cohesive situational awarenessviewpoint from local squad data. As weapon response device-based data isprovided, the edge compute resource clusters may process streaming datainto squad member locations, headings, gesture detections, threatlocations, and discharge/line of fire information. The edge computeresources may also provide an interface for consumers on the LAN 5512 toingest data from the edge compute resources 5560.

In embodiments, some or all of the weapon response devices 5210A, 52106and 5210C and the user devices 5610A, 5610B and 5610C can operate on amesh network. In this regard, when one weapon response device may be outof range from connecting directly to the mobile networking hub 5510,some or all of the weapon response devices can communicate and relaydata between each other such that the data can eventually reach themobile networking hub 5510. Further, such data is buffered on eachsoldier having the respective weapon response device. Additionally, datamay be stored locally (temporarily) on the user devices 5610A, 5610B and5610C in the event that some or all of the soldiers are out of range ofthe mobile networking hub 5510 and offloaded once within range. Inexamples, the data may be stored in flash memory 5306 or other storagedevice.

In embodiments user systems and devices and the like may be applicationssuch as Android Team Awareness Kit (ATAK) 4570 or other systems directlyconnected to the LAN network 4512, as shown in FIG. 62 . These may beprovided data through, for example, a streaming API available from anedge compute resource cluster.

Authentication may occur between a user application and an edge computeresource on connect, which then may enables edge compute resources'available data to be viewed or ingested into the user application. In anexample, this activity may be used by a squad leader using the ATAK/MIRAvisualization to view a near-real time squad-based situational awarenessdata provided by the edge compute resources. Data may be furthercommunicated to a consumer 5580 to provide date integration to otherdisconnected networks.

Detailed embodiments of the present disclosure are disclosed herein;however, it is to be understood that the disclosed embodiments aremerely exemplary of the present disclosure, which may be embodied invarious forms. Therefore, specific structural and functional detailsdisclosed herein are not to be interpreted as limiting, but merely as abasis for the claims and as a representative basis for teaching oneskilled in the art to variously employ the present disclosure invirtually any appropriately detailed structure.

While only a few embodiments of the present disclosure have been shownand described, it will be obvious to those skilled in the art that manychanges and modifications may be made thereunto without departing fromthe spirit and scope of the present disclosure as described in thefollowing claims. All patent applications and patents, both foreign anddomestic, and all other publications referenced herein are incorporatedherein in their entireties to the full extent permitted by law.

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software, program codes,and/or instructions on a processor. The present disclosure may beimplemented as a method on the machine, as a system or apparatus as partof or in relation to the machine, or as a computer program productembodied in a computer readable medium executing on one or more of themachines. In embodiments, the processor may be part of a server, cloudserver, client, network infrastructure, mobile computing platform,stationary computing platform, or other computing platforms. A processormay be any kind of computational or processing device capable ofexecuting program instructions, codes, binary instructions, and thelike. The processor may be or may include a signal processor, digitalprocessor, embedded processor, microprocessor or any variant such as aco-processor (math co-processor, graphic co-processor, communicationco-processor and the like) and the like that may directly or indirectlyfacilitate execution of program code or program instructions storedthereon. In addition, the processor may enable execution of multipleprograms, threads, and codes. The threads may be executed simultaneouslyto enhance the performance of the processor and to facilitatesimultaneous operations of the application. By way of implementation,methods, program codes, program instructions and the like describedherein may be implemented in one or more threads. The thread may spawnother threads that may have assigned priorities associated with them;the processor may execute these threads based on priority or any otherorder based on instructions provided in the program code. The processor,or any machine utilizing one, may include non-transitory memory thatstores methods, codes, instructions and programs as described herein andelsewhere. The processor may access a non-transitory storage mediumthrough an interface that may store methods, codes, and instructions asdescribed herein and elsewhere. The storage medium associated with theprocessor for storing methods, programs, codes, program instructions orother type of instructions capable of being executed by the computing orprocessing device may include but may not be limited to one or more of aCD-ROM, DVD, memory, hard disk, flash drive, RAM, ROM, cache, and thelike.

A processor may include one or more cores that may enhance speed andperformance of a multiprocessor. In embodiments, the process may be adual core processor, quad core processors, other chip-levelmultiprocessor and the like that combine two or more independent cores(called a die).

The methods and systems described herein may be deployed in part or inwhole through a machine that executes computer software on a server,client, firewall, gateway, hub, router, or other such computer and/ornetworking hardware. The software program may be associated with aserver that may include a file server, print server, domain server,internet server, intranet server, cloud server, and other variants suchas secondary server, host server, distributed server, and the like. Theserver may include one or more of memories, processors, computerreadable media, storage media, ports (physical and virtual),communication devices, and interfaces capable of accessing otherservers, clients, machines, and devices through a wired or a wirelessmedium, and the like. The methods, programs, or codes as describedherein and elsewhere may be executed by the server. In addition, otherdevices required for the execution of methods as described in thisapplication may be considered as a part of the infrastructure associatedwith the server.

The server may provide an interface to other devices including, withoutlimitation, clients, other servers, printers, database servers, printservers, file servers, communication servers, distributed servers,social networks, and the like. Additionally, this coupling and/orconnection may facilitate remote execution of program across thenetwork. The networking of some or all of these devices may facilitateparallel processing of a program or method at one or more locationswithout deviating from the scope of the present disclosure. In addition,any of the devices attached to the server through an interface mayinclude at least one storage medium capable of storing methods,programs, code and/or instructions. A central repository may provideprogram instructions to be executed on different devices. In thisimplementation, the remote repository may act as a storage medium forprogram code, instructions, and programs.

The software program may be associated with a client that may include afile client, print client, domain client, internet client, intranetclient and other variants such as secondary client, host client,distributed client, and the like. The client may include one or more ofmemories, processors, computer readable media, storage media, ports(physical and virtual), communication devices, and interfaces capable ofaccessing other clients, servers, machines, and devices through a wiredor a wireless medium, and the like. The methods, programs, or codes asdescribed herein and elsewhere may be executed by the client. Inaddition, other devices required for execution of methods as describedin this application may be considered as a part of the infrastructureassociated with the client.

The client may provide an interface to other devices including, withoutlimitation, servers, other clients, printers, database servers, printservers, file servers, communication servers, distributed servers, andthe like. Additionally, this coupling and/or connection may facilitateremote execution of program across the network. The networking of someor all of these devices may facilitate parallel processing of a programor method at one or more locations without deviating from the scope ofthe present disclosure. In addition, any of the devices attached to theclient through an interface may include at least one storage mediumcapable of storing methods, programs, applications, code and/orinstructions. A central repository may provide program instructions tobe executed on different devices. In this implementation, the remoterepository may act as a storage medium for program code, instructions,and programs.

The methods and systems described herein may be deployed in part or inwhole through network infrastructures. The network infrastructure mayinclude elements such as computing devices, servers, routers, hubs,firewalls, clients, personal computers, communication devices, routingdevices and other active and passive devices, modules and/or componentsas known in the art. The computing and/or non-computing device(s)associated with the network infrastructure may include, apart from othercomponents, a storage medium such as flash memory, buffer, stack, RAM,ROM, and the like. The processes, methods, program codes, instructionsdescribed herein and elsewhere may be executed by one or more of thenetwork infrastructural elements. The methods and systems describedherein may be adapted for use with any kind of private, community, orhybrid cloud computing network or cloud computing environment, includingthose which involve features of SaaS products, PaaS products, and/orinfrastructure as a service (IaaS) products.

The methods, program codes, and instructions described herein andelsewhere may be implemented on a cellular network having multiplecells. The cellular network may either be FDMA network or CDMA network.The cellular network may include mobile devices, cell sites, basestations, repeaters, antennas, towers, and the like. The cell networkmay be a GSM, GPRS, 3G, 4G, 5G, EVDO, mesh, or other networks types.

The methods, program codes, and instructions described herein andelsewhere may be implemented on or through mobile devices. The mobiledevices may include navigation devices, cell phones, mobile phones,PDAs, laptops, palmtops, netbooks, pagers, electronic book readers,music players and the like. These devices may include, apart from othercomponents, a storage medium such as a flash memory, buffer, RAM, ROMand one or more computing devices. The computing devices associated withmobile devices may be enabled to execute program codes, methods, andinstructions stored thereon. Alternatively, the mobile devices may beconfigured to execute instructions in collaboration with other devices.The mobile devices may communicate with base stations interfaced withservers and configured to execute program codes. The mobile devices maycommunicate on a peer-to-peer network, mesh network, or othercommunications network. The program code may be stored on the storagemedium associated with the server and executed by a computing deviceembedded within the server. The base station may include a computingdevice and a storage medium. The storage device may store program codesand instructions executed by the computing devices associated with thebase station.

The computer software, program codes, and/or instructions may be storedand/or accessed on machine readable media that may include: computercomponents, devices, and recording media that retain digital data usedfor computing for some interval of time; semiconductor storage known asRAM; mass storage typically for more permanent storage, such as opticaldiscs, forms of magnetic storage like hard disks, tapes, drums, cardsand other types; processor registers, cache memory, volatile memory,non-volatile memory; optical storage such as CD, DVD; removable mediasuch as flash memory (e.g. USB sticks or keys), floppy disks, magnetictape, paper tape, punch cards, standalone RAM disks, Zip drives,removable mass storage, off-line, and the like; other computer memorysuch as dynamic memory, static memory, read/write storage, mutablestorage, read only, random access, sequential access, locationaddressable, file addressable, content addressable, network attachedstorage, storage area network, bar codes, magnetic ink, and the like.

The methods and systems described herein may transform physical and/orintangible items from one state to another. The methods and systemsdescribed herein may also transform data representing physical and/orintangible items from one state to another.

The elements described and depicted herein, including in flow charts andblock diagrams throughout the figures, imply logical boundaries betweenthe elements. However, according to software or hardware engineeringpractices, the depicted elements and the functions thereof may beimplemented on machines through computer executable media having aprocessor capable of executing program instructions stored thereon as amonolithic software structure, as standalone software modules, or asmodules that employ external routines, code, services, and so forth, orany combination of these, and all such implementations may be within thescope of the present disclosure.

Examples of such machines may include, but may not be limited to, PDAs,laptops, personal computers, mobile phones, other handheld computingdevices, medical equipment, wired or wireless communication devices,transducers, chips, calculators, satellites, tablet PCs, electronicbooks, gadgets, electronic devices, devices having artificialintelligence, computing devices, networking equipment, servers, routers,and the like. Furthermore, the elements depicted in the flow chart andblock diagrams or any other logical component may be implemented on amachine capable of executing program instructions. Thus, while theforegoing drawings and descriptions set forth functional aspects of thedisclosed systems, no particular arrangement of software forimplementing these functional aspects should be inferred from thesedescriptions unless explicitly stated or otherwise clear from thecontext. Similarly, it will be appreciated that the various stepsidentified and described above may be varied, and that the order ofsteps may be adapted to particular applications of the techniquesdisclosed herein. All such variations and modifications are intended tofall within the scope of this disclosure. As such, the depiction and/ordescription of an order for various steps should not be understood torequire a particular order of execution for those steps, unless requiredby a particular application, or explicitly stated or otherwise clearfrom the context.

The methods and/or processes described above, and steps associatedtherewith, may be realized in hardware, software or any combination ofhardware and software suitable for a particular application. Thehardware may include a general-purpose computer and/or dedicatedcomputing device or specific computing device or particular aspect orcomponent of a specific computing device. The processes may be realizedin one or more microprocessors, microcontrollers, embeddedmicrocontrollers, programmable digital signal processors or otherprogrammable devices, along with internal and/or external memory. Theprocesses may also, or instead, be embodied in an application specificintegrated circuit, a programmable gate array, programmable array logic,or any other device or combination of devices that may be configured toprocess electronic signals. It will further be appreciated that one ormore of the processes may be realized as a computer executable codecapable of being executed on a machine-readable medium.

The computer executable code may be created using a structuredprogramming language such as C, an object oriented programming languagesuch as C++, or any other high-level or low-level programming language(including assembly languages, hardware description languages, anddatabase programming languages and technologies) that may be stored,compiled or interpreted to run on one of the above devices, as well asheterogeneous combinations of processors, processor architectures, orcombinations of different hardware and software, or any other machinecapable of executing program instructions.

Thus, in one aspect, methods described above and combinations thereofmay be embodied in computer executable code that, when executing on oneor more computing devices, performs the steps thereof. In anotheraspect, the methods may be embodied in systems that perform the stepsthereof and may be distributed across devices in a number of ways, orall of the functionality may be integrated into a dedicated, standalonedevice or other hardware. In another aspect, the means for performingthe steps associated with the processes described above may include anyof the hardware and/or software described above. All such permutationsand combinations are intended to fall within the scope of the presentdisclosure.

While the present disclosure has been disclosed in connection with thepreferred embodiments shown and described in detail, variousmodifications and improvements thereon will become readily apparent tothose skilled in the art. Accordingly, the spirit and scope of thepresent disclosure is not to be limited by the foregoing examples, butis to be understood in the broadest sense allowable by law.

The use of the terms “a,” “an,” “the,” and/or similar referents in thecontext of describing the present disclosure (especially in the contextof the following claims) is to be construed to cover both the singularand the plural unless otherwise indicated herein or clearly contradictedby context. The terms “comprising,” “having,” “including,” and/or“containing” are to be construed as open-ended terms (i.e., meaning“including, but not limited to”) unless otherwise noted. Recitations ofranges of values herein are merely intended to serve as a shorthandmethod of referring individually to each separate value falling withinthe range, unless otherwise indicated herein, and each separate value isincorporated into the specification as if it were individually recitedherein. All methods described herein can be performed in any suitableorder unless otherwise indicated herein or otherwise clearlycontradicted by context. The use of any and all examples, or exemplarylanguage (e.g., “such as”) provided herein, is intended merely to betterilluminate the present disclosure and does not pose a limitation on thescope of the present disclosure unless otherwise claimed. No language inthe specification should be construed as indicating any non-claimedelement as essential to the practice of the present disclosure.

While the foregoing written description enables one skilled in the artto make and use what is considered presently to be the best modethereof, those skill in the art will understand and appreciate theexistence of variations, combinations, and equivalents of the specificembodiment, method, and examples herein. The present disclosure shouldtherefore not be limited by the above-described embodiment, method, andexamples, but by all embodiments and methods within the scope and spiritof the present disclosure.

Any element in a claim that does not explicitly state “means for”performing a specified function, or “step for” performing a specifiedfunction, is not to be interpreted as a “means” or “step” clause asspecified in 35 U.S.C. § 112 (f). In particular, any use of “step of inthe claims is not intended to invoke the provision of 35 U.S.C. § 112(f).

Persons skilled in the art may appreciate that numerous designconfigurations may be possible to enjoy the functional benefits of theinventive systems. Thus, given the wide variety of configurations andarrangements of embodiments of the present disclosure the scope of theinventions are reflected by the breadth of the claims below rather thannarrowed by the embodiments described above.

What is claimed is:
 1. A method for recommending a corrective actionbased on a performance metric related to a discharge event of a firearm,the method comprising: receiving, by a first event detection moduleassociated with a first firearm of a first user, a plurality of firstinput signals over a sample window of time from a first inertialmeasurement unit (IMU) configured on the first firearm; identifying, bythe first event detection module, an occurrence of a first shotdischarge of the first firearm at a shot time based on the plurality offirst input signals; determining a first orientation of the firstfirearm at the shot time; determining a second orientation of the firstfirearm at a first time before the shot time; comparing the first andsecond orientations; determining a first performance metric related topre-shot weapon orientation; and displaying a first recommendedcorrective action to the first user based on the first performancemetric.
 2. The method of claim 2, further comprising: determining athird orientation of the first firearm at a second time after the shottime; comparing the first and third orientations; and determining asecond performance metric related to post-shot weapon orientation; anddisplaying a second recommended corrective action to the first userbased on the second performance metric.
 3. The method of claim 1,further comprising: establishing a historical baseline performancemetric of the first user based on collected historical shot detectioninformation about the first user; comparing the first performance metricwith the historical baseline performance metric; and displaying thecomparison.
 4. The method of claim 1 wherein receiving the plurality offirst input signals comprises: receiving a first acceleration inputsignal along a first axis, a second acceleration signal along a secondaxis and a third acceleration input signal along a third axis; andreceiving a first rotation input signal around the first axis, a secondrotation input signal around the second axis and a third accelerationinput signal around the third axis.
 5. The method of claim 1, furthercomprising: receiving, by a second event detection module associatedwith a second firearm of a second user, a plurality of second inputsignals over a sample window of time from a second IMU configured on thesecond firearm; identifying, by the second event detection module, anoccurrence of a first shot discharge of the second firearm at a shottime based on the plurality of second input signals; determining a firstorientation of the second firearm at the shot time; determining a secondorientation of the second firearm at a first time before the shot time;comparing the first and second orientations; and assigning a thirdperformance metric to the second user based on the first and secondorientations of the second firearm.
 6. The method of claim 5, furthercomprising: ranking a performance of the first user relative to thesecond user based on the first and third performance metrics; andproviding visual feedback to the first user of the ranking.
 7. Themethod of claim 6 wherein providing the visual feedback comprises:displaying the ranking on an augmented reality headset.
 8. The method ofclaim 5, further comprising: combining the first and third performancemetrics to a first group and assigning a first aggregate stability indexscore.
 9. The method of claim 6, further comprising: determining afourth performance metric for a third user; determining a fifthperformance metric for a fourth user; combining the fourth and fifthperformance metrics to a second group and assigning a second aggregateperformance index score; comparing the first aggregate performance indexscore with the second aggregate performance index score; and determininga ranked performance of the first group versus the second group based onthe comparing.
 10. The method of claim 1, wherein the plurality of firstinput signals include acceleration and rotation input signals andwherein identifying the occurrence of the first shot further comprises:assigning respective acceleration and rotation input signals to sampleevent candidates at respective windows of time; comparing theacceleration signals of the sampled event candidates to a dischargeacceleration template that represents a confirmed weapon dischargeevent; identifying a subset of accepted accelerations from the sampledevent candidates that satisfy the discharge acceleration template;comparing a first rotation input signal from the sample window of timeassociated with each accepted acceleration in the subset to a firstrotation template that represents a confirmed weapon discharge event;and determining whether the sampled event candidate is a discharge eventbased on satisfying both the discharge acceleration template and thefirst rotation template.
 11. A system for determining a performancemetric based on a discharge event of a firearm, the system comprising: afirst inertial measurement unit (IMU) disposed on a first firearm thatsenses movement including at least one of accelerations and rotations; afirst event detection module that receives a plurality of first inputsignals from the first IMU indicative of the movement and that isconfigured to: identify an occurrence of a first shot discharge of thefirst firearm at a shot time based on the plurality of first inputsignals; determine a first orientation of the first firearm at the shottime; determine a second orientation of the first firearm at a firsttime before the shot time; compare the first and second orientations;and determine a first performance metric related to pre-shot weaponorientation; and a display that displays the first recommendedcorrective action.
 12. The system of claim 11, wherein the first eventdetection module is further configured to: determine a third orientationof the first firearm at a second time after the shot time; compare thefirst and third orientations; determine a second performance metricrelated to post-shot weapon orientation; and display a secondrecommended corrective action to the first user based on the secondperformance metric.
 13. The system of claim 11 wherein the first eventdetection module is further configured to: establish a historicalbaseline performance metric of the first user based on collectedhistorical shot detection information about the first user; and comparethe first performance metric with the historical baseline performancemetric; wherein the comparison is shown on the display.
 14. The systemof claim 11 wherein the plurality of first input signals comprises: afirst acceleration input signal along a first axis, a secondacceleration signal along a second axis and a third acceleration inputsignal along a third axis; and a first rotation input signal around thefirst axis, a second rotation input signal around the second axis and athird acceleration input signal around the third axis.
 15. The system ofclaim 11, further comprising: a second inertial measurement unit (IMU)disposed on a second firearm that senses movement including at least oneof accelerations and rotations; a second event detection moduleassociated with the second firearm that receives a plurality of secondinput signals from the second IMU indicative of the movement and that isconfigured to: identify an occurrence of a first shot discharge of thesecond firearm at a shot time based on the plurality of second inputsignals; determine a first orientation of the second firearm at the shottime; determine a second orientation of the second firearm at a firsttime before the shot time; compare the first and second orientations;and assign a third performance metric to the second user based on thefirst and second orientations of the second firearm.
 16. The system ofclaim 15 wherein the first and third performance metrics are comparedand the display shows the comparison.
 17. The system of claim 14,wherein the display comprises a usage monitoring system including one ofa virtual reality and augmented reality system, wherein the firstperformance metric is displayed on the usage monitoring system.
 18. Thesystem of claim 15 wherein the first and third stability indexes arecombined to a first group and assigned a first aggregate performancemetric score.
 19. The system of claim 18, wherein the system is furtherconfigured to: determine a fourth stability index for a third user;determine a fifth stability index for a fourth user; combine the fourthand fifth stability indexes to a second group and assigning a secondaggregate stability index score to the second group; compare the firstaggregate performance metric score with the second aggregate performancemetric score; determine a ranked performance of the first group versusthe second group based on the comparing; and wherein the display showsthe ranked performance.
 20. The system of claim 11, wherein theplurality of first input signals include acceleration and rotation inputsignals and wherein the first event detection module is furtherconfigured to: assign respective acceleration and rotation input signalsto sample event candidates at respective windows of time; compare theacceleration signals of the sampled event candidates to a dischargeacceleration template that represents a confirmed weapon dischargeevent; identify a subset of accepted accelerations from the sampledevent candidates that satisfy the discharge acceleration template;compare a first rotation input signal from the sample window of timeassociated with each accepted acceleration in the subset to a firstrotation template that represents a confirmed weapon discharge event;and determine whether the sampled event candidate is a discharge eventbased on satisfying both the discharge acceleration template and thefirst rotation template.